-- Logs begin at Mon 2022-06-20 10:24:38 UTC, end at Mon 2022-06-20 11:33:59 UTC. -- Jun 20 10:50:19.750393 dv-cinder-846146-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 20 10:50:19.757778 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 20 10:50:19 2022] *** Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: nodename: dv-cinder-846146-3-iscsi Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: machine: x86_64 Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: clock source: unix Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: pcre jit disabled Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: detected number of CPU cores: 4 Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: current working directory: / Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: detected binary path: /usr/bin/uwsgi-core Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: your processes number limit is 47680 Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: your memory page size is 4096 bytes Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: detected max file descriptor number: 2048 Jun 20 10:50:19.760921 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: lock engine: pthread robust mutexes Jun 20 10:50:19.761819 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: thunder lock: enabled Jun 20 10:50:19.761819 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 20 10:50:19.761819 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 20 10:50:19.804974 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: Python main interpreter initialized at 0x555a554be9c0 Jun 20 10:50:19.804974 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: python threads support enabled Jun 20 10:50:19.804974 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: your server socket listen backlog is limited to 100 connections Jun 20 10:50:19.804974 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: your mercy for graceful operations on workers is 90 seconds Jun 20 10:50:19.805341 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: mapped 403077 bytes (393 KB) for 2 cores Jun 20 10:50:19.805414 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: *** Operational MODE: preforking *** Jun 20 10:50:19.805669 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: *** uWSGI is running in multiple interpreter mode *** Jun 20 10:50:19.805669 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: spawned uWSGI master process (pid: 80122) Jun 20 10:50:19.805895 dv-cinder-846146-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 20 10:50:19.806999 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: spawned uWSGI worker 1 (pid: 80132, cores: 1) Jun 20 10:50:19.806999 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: spawned uWSGI worker 2 (pid: 80133, cores: 1) Jun 20 10:50:19.806999 dv-cinder-846146-3-iscsi devstack@keystone.service[80122]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 20 10:50:25.240755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 10:50:25.240755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: from cryptography.utils import int_from_bytes Jun 20 10:50:25.241533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 10:50:25.241533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: from cryptography.utils import int_from_bytes Jun 20 10:50:25.242619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 10:50:25.242619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: from cryptography.utils import int_from_bytes Jun 20 10:50:25.243114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 10:50:25.243114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: from cryptography.utils import int_from_bytes Jun 20 10:50:25.290368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 20 10:50:25.290536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 20 10:50:25.290743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] command line args: [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 20 10:50:25.291013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 20 10:50:25.291168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ================================================================================ {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 20 10:50:25.291454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] admin_token = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.291697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] config_dir = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.291875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 20 10:50:25.292014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.292097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] config_source = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.292162 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 20 10:50:25.292235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] debug = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.292301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] command line args: [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 20 10:50:25.292488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.292708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 20 10:50:25.292708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ================================================================================ {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 20 10:50:25.292841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.292906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] admin_token = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.292962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.293167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] config_dir = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.293254 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.293315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.293466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.293548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] config_source = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.293614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] debug = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.293677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.293823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.293994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_config_append = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.293994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.294711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.294711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.294992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_dir = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.294992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_file = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.294992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.295306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.295385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.295444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.295517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.295636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.295636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_config_append = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.295753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.295825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.295935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.296011 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_dir = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.296080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.296151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_file = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.296261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.296376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.296456 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.296579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.296660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.296782 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.296862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.297009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.297103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.297216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.297301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.297378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.297578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.297710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.297850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.297937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.298010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.298155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.298244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.298493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.298671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.298765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.298845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] publish_errors = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.298922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.299006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.299180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.299315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.299399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.299476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.299643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.299883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] publish_errors = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] standard_threads = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300628 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300777 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] use_journal = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.300976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.301046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] use_json = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.301115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.301183 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] use_stderr = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.301340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] standard_threads = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.301474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] use_syslog = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.301552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.301622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.301787 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.301908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.302000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.302081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.302192 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] use_journal = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.302271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] use_json = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.302341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.302418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] use_stderr = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.302531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.302613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] use_syslog = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.302813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.302933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 10:50:25.303010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.303090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.303269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.303351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.303503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.303583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.303742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.303822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.303988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.304071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.304222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.304304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.304467 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.304553 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.304706 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.304788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.304939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.305031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.305180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.305260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.305422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.305503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.305650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.305729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.305894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.305982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.306161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.306253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.306378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.306466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.306618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.306699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.306850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.306932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.307097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.307185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.307333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.307409 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.307574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.307651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.307805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.307879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.308040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.308116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.308274 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.308352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.308511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.308594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.308753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.308829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.309006 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.309128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.309213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.309289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.309418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.309505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.309648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.309725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.309997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.310246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.310477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.310568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.310753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.310829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.310988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.311063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.311220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.311304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.311466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.311543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.311699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.311774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.311940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.312091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.312163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.312234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.312398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.312481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.312631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.312713 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.312864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.312940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.313092 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.313167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.313320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.313403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.313547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.313623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.313773 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.313848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.314004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.314115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.314262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.314339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.314491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.314572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.314724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.314835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.314956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.315057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.315188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.315308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.315438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.315546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.315643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.315747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.315874 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.315949 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.316087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.316173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.316323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.316399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.316555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.316646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.316786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.316871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.317019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.317103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.317247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.317325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.317484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.317565 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.317716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.317794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.317947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.318029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.318201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.318294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.318458 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.318544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.318692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.318770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.318921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.319000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.319154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.319241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.319384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.319460 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.319629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.319708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.319867 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.319946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.320103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.320182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.320346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.320421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.320578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.320656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.320810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.320888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.321044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.321120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.321283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.321361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.321537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.321711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.321871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.321989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.322134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.322214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.322357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.322446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.322596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.322674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.322840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.322919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.323079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.323159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.323324 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.323402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.323565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.323645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.323798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.323875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.324031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.324108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.324264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.324350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.324503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.324580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.324734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.324811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.324965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.325045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.325198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.325280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.325469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.325550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.325888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.326188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.326282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.326428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.326505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.326666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.326743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.326897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.326982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.external = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.327138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.327218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.327373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.327449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.327621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.327699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.327857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.327933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.password = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.328092 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] auth.token = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.328168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.328337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.328442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.328575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.328666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.328802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.external = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.328878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.329030 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.329115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.329278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.329356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.329528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.329613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.329753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.password = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.329830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.329981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.token = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.330058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.330242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.330321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.330476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.330564 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.330719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.330804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.330947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.331025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.331192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.331268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.331429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.331514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.331655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.331732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.331887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.331965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.332130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.332208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.332374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.332452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.332617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.332693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.332850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.332929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.333091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.333173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.333323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.333405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.333581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.333659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.333832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.333908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.334066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.334194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.334340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.334427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.334943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.335026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.335184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.335261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.335430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.335508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.335669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.335746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.335921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.336005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.336155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.336232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.336394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.336471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.336632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.336708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.336871 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.336949 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.337109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.337186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.337344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.337422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.337580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.337659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.337814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.337890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.338049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.338170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.338328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.338403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.338540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.338617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.338778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.338864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.339009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.339085 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.339253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.339330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.339484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.339569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.339716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.339794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.339946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.340022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.340187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.340271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.340412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.340489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.340647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.340724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.340893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.340979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.341226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.341380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.341500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.341782 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.342039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.342191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.342276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.342350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.342453 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.342545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.342684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.342771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.342919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.342997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.343159 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.343236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.343400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.343486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.343645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.343722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.343874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.343951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.344123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.344208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.344363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.344440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.344593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.344669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.344829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.344915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.345057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.345135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.345289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.345364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.345523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.345608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.345750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.345827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.345988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.346066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.346247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.346338 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.346487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.346564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.346724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.346802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.346960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.347211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.347289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.347459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.347545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.347690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.347768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.347917 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.347995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.348157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.348242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.348385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.348463 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.348621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.348699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.348853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.348929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.349088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.349165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.349315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.349392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.349547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.349623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.349792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.349870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.350034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.350128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.350289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.350366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.350516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.350604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.350762 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.350841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.350998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.351074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.351319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.351461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.351537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.351693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.351769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.351917 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.352002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.352146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.352228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.352391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.352469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.352620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.352705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.352864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.352941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.353107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.353184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.353350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.353435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.353578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.353655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.353805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.353883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.354038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.354158 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.354293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.354375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.354526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.354603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.354753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.354838 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.354999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.355075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.355245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.355321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.355483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.355569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.355722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.355799 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.355960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.356037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.356181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.356273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.356422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.356500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.356674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.356751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.356955 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.357186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.357430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.357552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.357676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.357798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.357920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.358050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.358204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.358324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.358446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.358568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.358690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.358822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.358945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.359068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.359189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.359313 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.359434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.359564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.359687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.359819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.359819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.360049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.360170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.360298 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.360420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.360540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.360540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] role.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.360769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.360769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.361006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] role.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.361128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.361249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.361376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.361507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.361629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.361824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.361950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.362950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.363020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.363170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.363249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.363415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] role.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.363502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.363662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] role.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.363740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.363893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.363970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.364142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.364312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.364390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.364556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.364634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.364813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.364900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.365064 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.365141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.365312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.365389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.365552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.365637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.365786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.365863 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.366015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.366135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.366284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.366377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.366533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.366741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.366819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.366985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.367063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.367220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.367306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.367469 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.367624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.367724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.367861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.367971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.368108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.368210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.368354 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.368432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.368600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.368686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.368844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.368922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.369078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.369156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.369329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.369415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.369571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.369648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.369804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.369881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.370038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.370134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.370294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.370372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.370545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.370669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.370798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.370921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.371055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.371178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.371301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.371423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.371553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.371676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.371799 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.371921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.372044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.372165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.372165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.372473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.372662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.372841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.372970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.373161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.373288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.373412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.373534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.connection = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.373665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.373796 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.373919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.374040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.374198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.374328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.374453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.374572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.374690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.374817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.374939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.375027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.375183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.375300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.375419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.375538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.375659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.375789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.375912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.376033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.376155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.376276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.376397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.376534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.376657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.376779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.376907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.377029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.377150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.connection = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.377283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.377417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.377529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.377648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.377771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.377891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.378009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.378009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80132) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 20 10:50:25.378311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.378311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.378536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.378657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.378807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.379029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.379268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.379511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.379746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.380073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.380444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.380734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.380996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.381266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.381526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 10:50:25.381710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 20 10:50:25.408726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.408966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.409180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.417758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.423453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.423686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.423916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.432513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.445675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.445889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.450944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.450944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.592034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.593225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.640778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.640930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.641148 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.641238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.704925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.705192 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.705329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.705411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.705548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.705628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.705689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.705805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.705882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.706004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.706102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.706171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.732048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.732259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.732433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.732518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.732582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.732703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.732781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.732853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.732927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.733049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.733166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.733244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.733316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.744738 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.745290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.757693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.757850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.758028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.758233 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.758322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.758463 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.758545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.758684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.758766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.758893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.758973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.759093 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.759174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80132) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.759302 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 10:50:25.777585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.777839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.777937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 10:50:25.778065 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 10:50:25.778248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.778330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.778448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 10:50:25.778530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 10:50:25.778679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.778797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.779007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.779224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.779316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.779575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.779753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.779905 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.790612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.790887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.791147 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.791425 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.793807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.794111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.794380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.794697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.801115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.802988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.804387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.804661 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.804939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.805220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.805493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.805721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.806005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.806282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.806409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.806569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.806753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.806844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.806969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.807089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.807213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.807335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.807597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.807867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.808139 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.808391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.808654 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.808919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.833409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.835367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.835639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.835897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.837505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.837783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.840822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.841249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.841499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.843164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.843495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.843776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.848749 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.849025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.851068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.851342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.853766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.854048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.854352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.854631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.854901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.855171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.855439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.855709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.855834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.856091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.856370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.856641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.856943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.857215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.857493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.857759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.876527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.876804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.878673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.878947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.881444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.883526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.885073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.885468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.887132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.887533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.890529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.892897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.894100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.896498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.897760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.898052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.898359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.898641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.898914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.899187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.900107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.900401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.900678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.900953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.901222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.901490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.918441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.918735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.919007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.919344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.921106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.921394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.921665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.921933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.927329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.929026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.929296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.929727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.931433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.931701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.932513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.932794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.933074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.933389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.933720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.935163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.935719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.936004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.936323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.936602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.947140 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.947397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.947711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.947988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.949690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.949830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.950006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.950214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.950305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.950370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.950734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.952420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.952714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.953293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.968647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.971353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.971951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.974623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.975280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.976998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.977876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.978654 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.978934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.979221 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.979596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.981218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.981493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.981815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.986629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.988231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.989083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.990707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:25.991477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.991755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.992146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.992427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.993875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.994189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.994574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:25.994845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.001797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:26.002081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.002382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.003889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:26.004171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.004448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.010207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80132) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:26.010484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.010761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.011036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.011305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.011687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.011972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.012195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 10:50:26.012321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.012459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.012540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.012732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.012872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.013104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.013231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.013330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.013462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.013762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.013907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.014049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80132) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.014198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.014484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.014751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.015016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.015286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.015550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.015818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.016088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 10:50:26.052491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x555a554be9c0 pid: 80132 (default app) Jun 20 10:50:26.054725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x555a554be9c0 pid: 80133 (default app) Jun 20 10:50:26.056009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce767e0-b98c-4339-a74c-68c0983c71c8 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:26.056303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce767e0-b98c-4339-a74c-68c0983c71c8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:26.056580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce767e0-b98c-4339-a74c-68c0983c71c8 None None] PATH_INFO: `/v3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:26.061158 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Mon Jun 20 10:50:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:33.818605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af763b0-4cc9-416a-a823-05af51aec62a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:33.819550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af763b0-4cc9-416a-a823-05af51aec62a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:33.819550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af763b0-4cc9-416a-a823-05af51aec62a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:33.826604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:33 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:33.832976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:33.833414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:33.833505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:33.911013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: INFO dbcounter [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] Registered counter for database keystone Jun 20 10:50:33.933203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG oslo_db.sqlalchemy.engines [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80132) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 20 10:50:33.943953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writer thread running {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 20 10:50:34.126534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG passlib.handlers.bcrypt [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80132) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 20 10:50:34.126859 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG passlib.handlers.bcrypt [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80132) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 20 10:50:34.182042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:34.183672 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:34.312895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 20 10:50:34.499293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af763b0-4cc9-416a-a823-05af51aec62a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:34.575429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: INFO dbcounter [None req-5af763b0-4cc9-416a-a823-05af51aec62a None None] Registered counter for database keystone Jun 20 10:50:34.596916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG oslo_db.sqlalchemy.engines [None req-5af763b0-4cc9-416a-a823-05af51aec62a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80133) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 20 10:50:34.607018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writer thread running {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 20 10:50:34.726371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af763b0-4cc9-416a-a823-05af51aec62a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:34.732029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10b9f6-23a7-47c2-af63-f4296d1a518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:34.732691 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10b9f6-23a7-47c2-af63-f4296d1a518a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:34.732971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10b9f6-23a7-47c2-af63-f4296d1a518a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:34.733246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10b9f6-23a7-47c2-af63-f4296d1a518a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:34.742428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10b9f6-23a7-47c2-af63-f4296d1a518a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:34.794195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6c10b9f6-23a7-47c2-af63-f4296d1a518a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 20 10:50:34.794493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10b9f6-23a7-47c2-af63-f4296d1a518a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:34.804764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Jun 20 10:50:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 10:50:34.814614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:34.834867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83f896-85a6-4b50-95b5-07025d16c684 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:34.840596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e153ae-f4ea-4622-902f-0d262ee30322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:34.841802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e153ae-f4ea-4622-902f-0d262ee30322 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:34.842113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e153ae-f4ea-4622-902f-0d262ee30322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:34.842394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e153ae-f4ea-4622-902f-0d262ee30322 admin admin] PATH_INFO: `/v3/projects/f76a7fac220343628fdd3080ad89317e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:34.846735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e153ae-f4ea-4622-902f-0d262ee30322 admin admin] RBAC: Authorizing `identity:get_project(project_id=f76a7fac220343628fdd3080ad89317e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:34.896569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: WARNING keystone.common.rbac_enforcer.enforcer [None req-20e153ae-f4ea-4622-902f-0d262ee30322 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 20 10:50:34.896909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e153ae-f4ea-4622-902f-0d262ee30322 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:34.901589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Mon Jun 20 10:50:34 2022] GET /identity/v3/projects/f76a7fac220343628fdd3080ad89317e => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:36.905101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdea3b-fb66-492a-8196-657056835bda None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:36.905923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdea3b-fb66-492a-8196-657056835bda None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:36.905923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdea3b-fb66-492a-8196-657056835bda None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:36.912745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:36.919517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483bf2-52da-4101-a6fe-c5eed46a4484 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:36.919952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483bf2-52da-4101-a6fe-c5eed46a4484 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:36.920054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483bf2-52da-4101-a6fe-c5eed46a4484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:36.940854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0c483bf2-52da-4101-a6fe-c5eed46a4484 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:36.942432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0c483bf2-52da-4101-a6fe-c5eed46a4484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:36.952589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 20 10:50:36.987091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdea3b-fb66-492a-8196-657056835bda None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:37.002322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdea3b-fb66-492a-8196-657056835bda None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:37.008353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc9d4a-ffd0-43d0-9138-07f5bdc13d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:37.008994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc9d4a-ffd0-43d0-9138-07f5bdc13d71 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:37.009276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc9d4a-ffd0-43d0-9138-07f5bdc13d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:37.009553 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc9d4a-ffd0-43d0-9138-07f5bdc13d71 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:37.018412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc9d4a-ffd0-43d0-9138-07f5bdc13d71 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:37.020210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc9d4a-ffd0-43d0-9138-07f5bdc13d71 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:37.029685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Jun 20 10:50:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 10:50:37.035424 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c483bf2-52da-4101-a6fe-c5eed46a4484 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:37.045587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c483bf2-52da-4101-a6fe-c5eed46a4484 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:37.051938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f1411-a3fd-45b5-ba5e-fc1c71585dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:37.052819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f1411-a3fd-45b5-ba5e-fc1c71585dca admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:37.053100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f1411-a3fd-45b5-ba5e-fc1c71585dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:37.053380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f1411-a3fd-45b5-ba5e-fc1c71585dca admin admin] PATH_INFO: `/v3/users/52894fe1eaba4d88aa400c34b4d5791b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:37.055041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f1411-a3fd-45b5-ba5e-fc1c71585dca admin admin] RBAC: Authorizing `identity:get_user(user_id=52894fe1eaba4d88aa400c34b4d5791b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:37.056906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f1411-a3fd-45b5-ba5e-fc1c71585dca admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:37.058514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Mon Jun 20 10:50:37 2022] GET /identity/v3/users/52894fe1eaba4d88aa400c34b4d5791b => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:39.179733 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f5364-4e10-43ce-906c-123ded638df5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:39.180659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f5364-4e10-43ce-906c-123ded638df5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:39.180918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f5364-4e10-43ce-906c-123ded638df5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:39.183340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:39.187594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5bd6b-8298-4ffd-ae09-46e5ee6a3e98 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:39.188113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5bd6b-8298-4ffd-ae09-46e5ee6a3e98 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:39.188252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5bd6b-8298-4ffd-ae09-46e5ee6a3e98 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:39.190165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbbae4-d3ff-4393-9213-acb7e635eebb None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:39.190644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbbae4-d3ff-4393-9213-acb7e635eebb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:39.190789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:39.190930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbbae4-d3ff-4393-9213-acb7e635eebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:39.196771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92f314-5d5f-4fab-8878-9455724fc06a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:39.197230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92f314-5d5f-4fab-8878-9455724fc06a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:39.197683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92f314-5d5f-4fab-8878-9455724fc06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:39.216635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bf92f314-5d5f-4fab-8878-9455724fc06a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:39.217833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bf92f314-5d5f-4fab-8878-9455724fc06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:39.237869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG passlib.handlers.bcrypt [None req-96bbbae4-d3ff-4393-9213-acb7e635eebb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80133) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 20 10:50:39.238329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG passlib.handlers.bcrypt [None req-96bbbae4-d3ff-4393-9213-acb7e635eebb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80133) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 20 10:50:39.285745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-96bbbae4-d3ff-4393-9213-acb7e635eebb None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:39.287407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-96bbbae4-d3ff-4393-9213-acb7e635eebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:39.314007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 7/12] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:39.336353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92f314-5d5f-4fab-8878-9455724fc06a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:39.360225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:39.395426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 6/13] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:39.418722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbbae4-d3ff-4393-9213-acb7e635eebb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:39.441701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92f314-5d5f-4fab-8878-9455724fc06a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:39.476954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d64c8-ad93-4571-b0e8-ca60a610707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:39.477614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d64c8-ad93-4571-b0e8-ca60a610707d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:39.477872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d64c8-ad93-4571-b0e8-ca60a610707d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:39.478212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d64c8-ad93-4571-b0e8-ca60a610707d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:39.520613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbbae4-d3ff-4393-9213-acb7e635eebb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:39.522931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:39.547762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d64c8-ad93-4571-b0e8-ca60a610707d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:39.549273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d64c8-ad93-4571-b0e8-ca60a610707d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:39.557747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aed1a6-25b0-42a4-9458-d121edb25195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:39.558856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aed1a6-25b0-42a4-9458-d121edb25195 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:39.559139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aed1a6-25b0-42a4-9458-d121edb25195 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:39.559504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aed1a6-25b0-42a4-9458-d121edb25195 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:39.572795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aed1a6-25b0-42a4-9458-d121edb25195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:39.575784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-37aed1a6-25b0-42a4-9458-d121edb25195 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:39.585001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:39.586137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:50:39.587134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:50:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:39.592645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aed1a6-25b0-42a4-9458-d121edb25195 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:39.657722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 8/15] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:50:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:50:39.663722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d64c8-ad93-4571-b0e8-ca60a610707d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:39.696031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aed1a6-25b0-42a4-9458-d121edb25195 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:39.728206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e7d32-a026-4d9b-9516-72fbacb464a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:39.728900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e7d32-a026-4d9b-9516-72fbacb464a2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:39.729181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e7d32-a026-4d9b-9516-72fbacb464a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:39.729455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e7d32-a026-4d9b-9516-72fbacb464a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:39.730246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e7d32-a026-4d9b-9516-72fbacb464a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:39.730551 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e7d32-a026-4d9b-9516-72fbacb464a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:39.732051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e7d32-a026-4d9b-9516-72fbacb464a2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:39.744331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 8/16] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:50:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:50:39.750062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e7d32-a026-4d9b-9516-72fbacb464a2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:39.764816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d64c8-ad93-4571-b0e8-ca60a610707d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:39.778418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:39.788219 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:39.800271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7038c-7c45-4d9b-81e9-3ef6cea667c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:39.801093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7038c-7c45-4d9b-81e9-3ef6cea667c5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:39.801381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7038c-7c45-4d9b-81e9-3ef6cea667c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:39.801663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7038c-7c45-4d9b-81e9-3ef6cea667c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:39.811788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7038c-7c45-4d9b-81e9-3ef6cea667c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:39.813287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7038c-7c45-4d9b-81e9-3ef6cea667c5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-92a7038c-7c45-4d9b-81e9-3ef6cea667c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:50:39.834213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:39.835414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:39.836192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:50:39.836192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:50:39.836192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:50:39.836192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:50:39.836192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:50:39.841829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7038c-7c45-4d9b-81e9-3ef6cea667c5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:39.852326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e7d32-a026-4d9b-9516-72fbacb464a2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:39.888887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890abbaa-04a1-4661-ae11-e6f70301b61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:39.889766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890abbaa-04a1-4661-ae11-e6f70301b61c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:39.890046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890abbaa-04a1-4661-ae11-e6f70301b61c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:39.890352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890abbaa-04a1-4661-ae11-e6f70301b61c None admin] PATH_INFO: `/v3/users/52894fe1eaba4d88aa400c34b4d5791b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:39.891915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890abbaa-04a1-4661-ae11-e6f70301b61c None admin] RBAC: Authorizing `identity:get_user(user_id=52894fe1eaba4d88aa400c34b4d5791b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:39.893379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890abbaa-04a1-4661-ae11-e6f70301b61c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:39.895048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:50:39 2022] GET /identity/v3/users/52894fe1eaba4d88aa400c34b4d5791b => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:39.900752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890abbaa-04a1-4661-ae11-e6f70301b61c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:39.946975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7038c-7c45-4d9b-81e9-3ef6cea667c5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:39.981849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4c664-b949-4134-9f0c-667931de8cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:39.982557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4c664-b949-4134-9f0c-667931de8cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:39.982840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4c664-b949-4134-9f0c-667931de8cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:39.983119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4c664-b949-4134-9f0c-667931de8cb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:39.983937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4c664-b949-4134-9f0c-667931de8cb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:39.984239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4c664-b949-4134-9f0c-667931de8cb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:39.985661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4c664-b949-4134-9f0c-667931de8cb6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:39.998409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:40.003293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:40.005593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890abbaa-04a1-4661-ae11-e6f70301b61c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:40.041358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c471632-601c-47f4-a0c9-c9a21824fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:40.042221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c471632-601c-47f4-a0c9-c9a21824fa6a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:40.042576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c471632-601c-47f4-a0c9-c9a21824fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:40.042856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c471632-601c-47f4-a0c9-c9a21824fa6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:40.044218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c471632-601c-47f4-a0c9-c9a21824fa6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:40.045697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c471632-601c-47f4-a0c9-c9a21824fa6a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:40.047443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:50:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:50:40.053572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4c664-b949-4134-9f0c-667931de8cb6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:40.092481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:40.164744 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4c664-b949-4134-9f0c-667931de8cb6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:40.202000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf674ec-38b1-4aac-aada-99a6ebc1a99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:40.202647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf674ec-38b1-4aac-aada-99a6ebc1a99a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:40.202898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf674ec-38b1-4aac-aada-99a6ebc1a99a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:40.203182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf674ec-38b1-4aac-aada-99a6ebc1a99a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:40.204035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf674ec-38b1-4aac-aada-99a6ebc1a99a None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.domain.id=default, user.id=52894fe1eaba4d88aa400c34b4d5791b) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:40.204353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf674ec-38b1-4aac-aada-99a6ebc1a99a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:40.205795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf674ec-38b1-4aac-aada-99a6ebc1a99a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:40.213156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Jun 20 10:50:40 2022] GET /identity/v3/role_assignments?user.id=52894fe1eaba4d88aa400c34b4d5791b&scope.domain.id=default&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:42.396010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a058eeb-b700-45d3-8b16-053fba971423 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:42.397028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a058eeb-b700-45d3-8b16-053fba971423 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:42.397607 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a058eeb-b700-45d3-8b16-053fba971423 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:42.399227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:42.408049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e9b90-9da1-499f-815a-7c1349e64c96 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:42.408894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e9b90-9da1-499f-815a-7c1349e64c96 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:42.409358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e9b90-9da1-499f-815a-7c1349e64c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:42.435653 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-235e9b90-9da1-499f-815a-7c1349e64c96 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:42.437161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-235e9b90-9da1-499f-815a-7c1349e64c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:42.525649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 12/23] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:42.561320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a058eeb-b700-45d3-8b16-053fba971423 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:42.657818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbeb4a-9ec0-4f8e-961e-6326f88f3f74 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:42.658449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbeb4a-9ec0-4f8e-961e-6326f88f3f74 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:42.658915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbeb4a-9ec0-4f8e-961e-6326f88f3f74 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:42.660480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 13/24] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:50:42.667350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898617ea-9047-4551-af93-ea47b9228adb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:42.667879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898617ea-9047-4551-af93-ea47b9228adb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:42.668373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898617ea-9047-4551-af93-ea47b9228adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:42.707136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-898617ea-9047-4551-af93-ea47b9228adb None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:42.708689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-898617ea-9047-4551-af93-ea47b9228adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:42.711118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a058eeb-b700-45d3-8b16-053fba971423 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:42.756749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:42.771274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474a070-e382-4fc6-b4f8-f2b7e7169735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:42.772336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474a070-e382-4fc6-b4f8-f2b7e7169735 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:42.772622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474a070-e382-4fc6-b4f8-f2b7e7169735 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:42.773298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474a070-e382-4fc6-b4f8-f2b7e7169735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:42.794753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474a070-e382-4fc6-b4f8-f2b7e7169735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:42.796212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e474a070-e382-4fc6-b4f8-f2b7e7169735 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e474a070-e382-4fc6-b4f8-f2b7e7169735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:50:42.818114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:42.818947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:42.819745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:50:42.819745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:50:42.819745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:50:42.819745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:50:42.819745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 12/25] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:42.822115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05354539-ea93-49d0-8aac-6b681b5705b8 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:42.822115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05354539-ea93-49d0-8aac-6b681b5705b8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:42.822115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05354539-ea93-49d0-8aac-6b681b5705b8 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:42.827805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 13/26] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:42.829220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05354539-ea93-49d0-8aac-6b681b5705b8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:42.852409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:42.870678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 14/27] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:42.873822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512c76d-d643-41b4-95f6-b4008a866219 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:42.874134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512c76d-d643-41b4-95f6-b4008a866219 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:42.874421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512c76d-d643-41b4-95f6-b4008a866219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:42.903360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6512c76d-d643-41b4-95f6-b4008a866219 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:42.904585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6512c76d-d643-41b4-95f6-b4008a866219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:42.985351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 15/28] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:42.986808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512c76d-d643-41b4-95f6-b4008a866219 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:43.047988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05354539-ea93-49d0-8aac-6b681b5705b8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:43.119626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77a3b-3845-45c5-831f-4bbb90aa3a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:43.120583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77a3b-3845-45c5-831f-4bbb90aa3a4a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:43.122214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77a3b-3845-45c5-831f-4bbb90aa3a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:43.123532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77a3b-3845-45c5-831f-4bbb90aa3a4a None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:43.124972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c77a3b-3845-45c5-831f-4bbb90aa3a4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:43.126117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c77a3b-3845-45c5-831f-4bbb90aa3a4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:43.128860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c77a3b-3845-45c5-831f-4bbb90aa3a4a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:43.164613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:43.170163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 14/29] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:50:43.171488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77a3b-3845-45c5-831f-4bbb90aa3a4a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:43.192151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512c76d-d643-41b4-95f6-b4008a866219 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:43.252021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc421d8-df6d-4c9a-800b-a359ee5dd066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:43.253085 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc421d8-df6d-4c9a-800b-a359ee5dd066 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:43.253580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc421d8-df6d-4c9a-800b-a359ee5dd066 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:43.254044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc421d8-df6d-4c9a-800b-a359ee5dd066 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:43.255055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc421d8-df6d-4c9a-800b-a359ee5dd066 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:43.256860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc421d8-df6d-4c9a-800b-a359ee5dd066 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:43.293003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:43.322470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77a3b-3845-45c5-831f-4bbb90aa3a4a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:43.364818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28a504-4f92-4903-9f1e-b21f83c19c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:43.365883 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28a504-4f92-4903-9f1e-b21f83c19c3b None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:43.366410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28a504-4f92-4903-9f1e-b21f83c19c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:43.366893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28a504-4f92-4903-9f1e-b21f83c19c3b None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:43.367848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28a504-4f92-4903-9f1e-b21f83c19c3b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:43.369437 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28a504-4f92-4903-9f1e-b21f83c19c3b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:43.404906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 16/30] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jun 20 10:50:42 2022] POST /identity/v3/roles => generated 225 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:50:43.411125 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc421d8-df6d-4c9a-800b-a359ee5dd066 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:43.504416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 15/31] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jun 20 10:50:43 2022] POST /identity/v3/roles => generated 219 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:50:43.507057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915cc71-4996-40a5-b01b-de6d470ed2c5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:43.507600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915cc71-4996-40a5-b01b-de6d470ed2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:43.508105 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915cc71-4996-40a5-b01b-de6d470ed2c5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:43.510076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 16/32] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:43.522892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959dfd-0ed8-4c8c-83c1-f86ed73f527e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:43.523514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959dfd-0ed8-4c8c-83c1-f86ed73f527e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:43.523992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959dfd-0ed8-4c8c-83c1-f86ed73f527e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:43.543873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b9959dfd-0ed8-4c8c-83c1-f86ed73f527e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:43.545949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b9959dfd-0ed8-4c8c-83c1-f86ed73f527e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:43.626566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc421d8-df6d-4c9a-800b-a359ee5dd066 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:43.638656 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 17/33] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:50:43.640524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebaad7-5532-4940-a1cb-5568d4439142 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:43.641044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebaad7-5532-4940-a1cb-5568d4439142 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:43.641518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebaad7-5532-4940-a1cb-5568d4439142 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:43.647585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 18/34] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:43.654795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c01c5-d7e6-4fc6-b087-e4ad27f5a089 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:43.655512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c01c5-d7e6-4fc6-b087-e4ad27f5a089 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:43.655991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c01c5-d7e6-4fc6-b087-e4ad27f5a089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:43.676834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-385c01c5-d7e6-4fc6-b087-e4ad27f5a089 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:43.678581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-385c01c5-d7e6-4fc6-b087-e4ad27f5a089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:43.714866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf07d2-c056-4129-92bc-4aa98f550070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:43.715598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf07d2-c056-4129-92bc-4aa98f550070 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:43.715893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf07d2-c056-4129-92bc-4aa98f550070 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:43.716173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf07d2-c056-4129-92bc-4aa98f550070 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:43.716925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf07d2-c056-4129-92bc-4aa98f550070 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:43.718778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf07d2-c056-4129-92bc-4aa98f550070 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:43.748138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:43.763002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:43.763353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 17/35] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:50:43 2022] POST /identity/v3/projects => generated 301 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:50:43.764876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf07d2-c056-4129-92bc-4aa98f550070 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:43.773195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 19/36] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:50:43.810276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c01c5-d7e6-4fc6-b087-e4ad27f5a089 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:43.892528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf07d2-c056-4129-92bc-4aa98f550070 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:43.946131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377382c-3412-470d-8c66-aaf344a3d45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:43.946868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c01c5-d7e6-4fc6-b087-e4ad27f5a089 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:43.947066 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377382c-3412-470d-8c66-aaf344a3d45c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:43.947332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377382c-3412-470d-8c66-aaf344a3d45c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:43.947613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377382c-3412-470d-8c66-aaf344a3d45c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:43.962795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377382c-3412-470d-8c66-aaf344a3d45c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-4377382c-3412-470d-8c66-aaf344a3d45c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:43.974019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:43.974922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:50:43.975667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 18/37] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:50:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:43.983473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377382c-3412-470d-8c66-aaf344a3d45c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:43.994562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4a739-c2cb-4cd6-8d94-815d6b2ab247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:43.995375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4a739-c2cb-4cd6-8d94-815d6b2ab247 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:43.995667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4a739-c2cb-4cd6-8d94-815d6b2ab247 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:43.995945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4a739-c2cb-4cd6-8d94-815d6b2ab247 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:43.997401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4a739-c2cb-4cd6-8d94-815d6b2ab247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:43.999264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4a739-c2cb-4cd6-8d94-815d6b2ab247 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:44.001124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 20/38] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:50:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:50:44.006567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4a739-c2cb-4cd6-8d94-815d6b2ab247 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:44.061160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:44.092397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:44.108466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377382c-3412-470d-8c66-aaf344a3d45c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:44.137151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4a739-c2cb-4cd6-8d94-815d6b2ab247 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:44.144654 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e192b-2041-464d-844b-a0740fb0261c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:44.145348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e192b-2041-464d-844b-a0740fb0261c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:44.145638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e192b-2041-464d-844b-a0740fb0261c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:44.145924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e192b-2041-464d-844b-a0740fb0261c None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:44.146721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e192b-2041-464d-844b-a0740fb0261c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:44.147040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e192b-2041-464d-844b-a0740fb0261c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:44.148463 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e192b-2041-464d-844b-a0740fb0261c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:44.160779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 19/39] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:50:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:50:44.166384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e192b-2041-464d-844b-a0740fb0261c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:44.178451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44987a-224d-4be0-94b4-d27eaa5baf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:44.179132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44987a-224d-4be0-94b4-d27eaa5baf46 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:44.179501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44987a-224d-4be0-94b4-d27eaa5baf46 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:44.179775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44987a-224d-4be0-94b4-d27eaa5baf46 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:44.180639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44987a-224d-4be0-94b4-d27eaa5baf46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:44.182183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44987a-224d-4be0-94b4-d27eaa5baf46 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:44.211480 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 21/40] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:50:44 2022] POST /identity/v3/projects => generated 312 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 10:50:44.213018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bafc3-338d-4afa-9815-95c4398dd2b1 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:44.213309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bafc3-338d-4afa-9815-95c4398dd2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:44.213588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bafc3-338d-4afa-9815-95c4398dd2b1 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:44.214868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 22/41] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:44.226025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce228fa-ecea-4c1c-85a8-e846f0c9e7d6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:44.226889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce228fa-ecea-4c1c-85a8-e846f0c9e7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:44.227175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce228fa-ecea-4c1c-85a8-e846f0c9e7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:44.244973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-dce228fa-ecea-4c1c-85a8-e846f0c9e7d6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:44.246394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dce228fa-ecea-4c1c-85a8-e846f0c9e7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:44.287580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e192b-2041-464d-844b-a0740fb0261c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:44.325635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 23/42] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:50:44.349385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce228fa-ecea-4c1c-85a8-e846f0c9e7d6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:44.406922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:44.431718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:44.444986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56bb10-da19-4063-9bf2-321e25892c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:44.445884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56bb10-da19-4063-9bf2-321e25892c9e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:44.446206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56bb10-da19-4063-9bf2-321e25892c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:44.446532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56bb10-da19-4063-9bf2-321e25892c9e None admin] PATH_INFO: `/v3/users/52894fe1eaba4d88aa400c34b4d5791b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:44.447795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56bb10-da19-4063-9bf2-321e25892c9e None admin] RBAC: Authorizing `identity:get_user(user_id=52894fe1eaba4d88aa400c34b4d5791b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:44.449287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56bb10-da19-4063-9bf2-321e25892c9e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:44.451337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 20/43] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:50:44 2022] GET /identity/v3/users/52894fe1eaba4d88aa400c34b4d5791b => generated 247 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:44.456873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56bb10-da19-4063-9bf2-321e25892c9e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:44.470486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce228fa-ecea-4c1c-85a8-e846f0c9e7d6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:44.509887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6f322-f33c-4127-afa1-eb47c9ac7943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:44.510700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6f322-f33c-4127-afa1-eb47c9ac7943 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:44.510989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6f322-f33c-4127-afa1-eb47c9ac7943 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:44.511270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6f322-f33c-4127-afa1-eb47c9ac7943 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:44.512037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6f322-f33c-4127-afa1-eb47c9ac7943 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:44.513646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6f322-f33c-4127-afa1-eb47c9ac7943 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:44.527809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 24/44] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jun 20 10:50:44 2022] POST /identity/v3/roles => generated 223 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 10:50:44.578723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56bb10-da19-4063-9bf2-321e25892c9e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:44.620124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41a48c-a519-4263-b59b-f319242724ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:44.621401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41a48c-a519-4263-b59b-f319242724ff None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:44.622114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41a48c-a519-4263-b59b-f319242724ff None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:44.622700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41a48c-a519-4263-b59b-f319242724ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:44.624222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41a48c-a519-4263-b59b-f319242724ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:44.625978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41a48c-a519-4263-b59b-f319242724ff None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:44.627968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 21/45] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:50:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:44.633819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6f322-f33c-4127-afa1-eb47c9ac7943 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:44.746564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6f322-f33c-4127-afa1-eb47c9ac7943 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:44.749603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:44.780565 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15940d45-c724-4cd5-8dce-3f46d9c2578b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:44.781452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15940d45-c724-4cd5-8dce-3f46d9c2578b None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:44.781735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15940d45-c724-4cd5-8dce-3f46d9c2578b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:44.782010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15940d45-c724-4cd5-8dce-3f46d9c2578b None admin] PATH_INFO: `/v3/domains/default/users/52894fe1eaba4d88aa400c34b4d5791b/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:44.784073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15940d45-c724-4cd5-8dce-3f46d9c2578b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=52894fe1eaba4d88aa400c34b4d5791b, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:44.785550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15940d45-c724-4cd5-8dce-3f46d9c2578b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:44.799483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 25/46] 10.222.0.33 () {58 vars in 1361 bytes} [Mon Jun 20 10:50:44 2022] PUT /identity/v3/domains/default/users/52894fe1eaba4d88aa400c34b4d5791b/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:50:46.257987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f1cd9-d8e0-498a-98ca-8811461bfb46 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:46.258521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f1cd9-d8e0-498a-98ca-8811461bfb46 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:46.258521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f1cd9-d8e0-498a-98ca-8811461bfb46 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:46.259870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 22/47] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:46.265714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2dc65-76a6-48a6-9a3c-fd677a514adf None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:46.266140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2dc65-76a6-48a6-9a3c-fd677a514adf None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:46.266430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2dc65-76a6-48a6-9a3c-fd677a514adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:46.283945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-59f2dc65-76a6-48a6-9a3c-fd677a514adf None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:46.285153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-59f2dc65-76a6-48a6-9a3c-fd677a514adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:46.354489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 26/48] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:46.378071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f1cd9-d8e0-498a-98ca-8811461bfb46 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:46.443142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:46.488322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f1cd9-d8e0-498a-98ca-8811461bfb46 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:46.523264 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7274e-77a1-438e-8a7e-05a6ede5471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:46.523991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7274e-77a1-438e-8a7e-05a6ede5471c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:46.524274 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7274e-77a1-438e-8a7e-05a6ede5471c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:46.524552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7274e-77a1-438e-8a7e-05a6ede5471c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:46.525764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7274e-77a1-438e-8a7e-05a6ede5471c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:46.527256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7274e-77a1-438e-8a7e-05a6ede5471c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:46.529219 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 23/49] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:50:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:46.534935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2dc65-76a6-48a6-9a3c-fd677a514adf None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:46.630627 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:46.638532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2dc65-76a6-48a6-9a3c-fd677a514adf None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:46.673937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c11c8b-d403-46c9-bafc-60c951f0d442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:46.674641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c11c8b-d403-46c9-bafc-60c951f0d442 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:46.674921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c11c8b-d403-46c9-bafc-60c951f0d442 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:46.675198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c11c8b-d403-46c9-bafc-60c951f0d442 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:46.676223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c11c8b-d403-46c9-bafc-60c951f0d442 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:46.677700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c11c8b-d403-46c9-bafc-60c951f0d442 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:46.704717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 27/50] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Jun 20 10:50:46 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 10:50:46.770482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84101b81-46f6-404f-ae68-c25b2f442773 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:46.770782 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84101b81-46f6-404f-ae68-c25b2f442773 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:46.771048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84101b81-46f6-404f-ae68-c25b2f442773 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:46.772461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 24/51] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:46.778563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245b826-7516-465b-a200-2ffdc06dfae9 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:46.778900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245b826-7516-465b-a200-2ffdc06dfae9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:46.779186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245b826-7516-465b-a200-2ffdc06dfae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:46.798982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8245b826-7516-465b-a200-2ffdc06dfae9 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:46.800296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8245b826-7516-465b-a200-2ffdc06dfae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:46.983454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 28/52] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:47.011667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84101b81-46f6-404f-ae68-c25b2f442773 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:47.119472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84101b81-46f6-404f-ae68-c25b2f442773 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:47.144752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:47.154451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722ff87-32d9-4fc8-b157-fd5db61eb311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:47.155260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722ff87-32d9-4fc8-b157-fd5db61eb311 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:47.155548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722ff87-32d9-4fc8-b157-fd5db61eb311 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:47.155828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722ff87-32d9-4fc8-b157-fd5db61eb311 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:47.166929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722ff87-32d9-4fc8-b157-fd5db61eb311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-b722ff87-32d9-4fc8-b157-fd5db61eb311 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:47.174592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:47.175440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:50:47.176410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 25/53] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:50:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:47.181851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245b826-7516-465b-a200-2ffdc06dfae9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:47.264066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:47.289752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245b826-7516-465b-a200-2ffdc06dfae9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:47.327215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1c33c-fada-4dfc-ba0d-f16d6d8c106e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:47.327886 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1c33c-fada-4dfc-ba0d-f16d6d8c106e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:47.328172 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1c33c-fada-4dfc-ba0d-f16d6d8c106e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:47.328449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1c33c-fada-4dfc-ba0d-f16d6d8c106e None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:47.329209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1c33c-fada-4dfc-ba0d-f16d6d8c106e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:47.329519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1c33c-fada-4dfc-ba0d-f16d6d8c106e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:47.330973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1c33c-fada-4dfc-ba0d-f16d6d8c106e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:47.340781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 29/54] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:50:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:47.347043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722ff87-32d9-4fc8-b157-fd5db61eb311 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:47.451722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722ff87-32d9-4fc8-b157-fd5db61eb311 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:47.487730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2e063-d76d-4663-8a3f-eba5765304e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:47.488617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2e063-d76d-4663-8a3f-eba5765304e5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:47.488905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2e063-d76d-4663-8a3f-eba5765304e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:47.489184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2e063-d76d-4663-8a3f-eba5765304e5 None admin] PATH_INFO: `/v3/users/52894fe1eaba4d88aa400c34b4d5791b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:47.490601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2e063-d76d-4663-8a3f-eba5765304e5 None admin] RBAC: Authorizing `identity:get_user(user_id=52894fe1eaba4d88aa400c34b4d5791b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:47.492090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2e063-d76d-4663-8a3f-eba5765304e5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:47.493761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 26/55] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:50:47 2022] GET /identity/v3/users/52894fe1eaba4d88aa400c34b4d5791b => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:47.499704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1c33c-fada-4dfc-ba0d-f16d6d8c106e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:47.603633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1c33c-fada-4dfc-ba0d-f16d6d8c106e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:47.639766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba79fa4-1ce5-4c7c-a04e-22fc8393d1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:47.640561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79fa4-1ce5-4c7c-a04e-22fc8393d1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:47.640840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79fa4-1ce5-4c7c-a04e-22fc8393d1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:47.641113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba79fa4-1ce5-4c7c-a04e-22fc8393d1b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:47.642353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba79fa4-1ce5-4c7c-a04e-22fc8393d1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:47.643801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba79fa4-1ce5-4c7c-a04e-22fc8393d1b2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:47.645449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 30/56] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:50:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:47.651392 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2e063-d76d-4663-8a3f-eba5765304e5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:47.710401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:47.756146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2e063-d76d-4663-8a3f-eba5765304e5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:47.791516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a220ca0-cac6-4d2b-9ef4-fe1ea1578a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:47.792178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a220ca0-cac6-4d2b-9ef4-fe1ea1578a2f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:47.792462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a220ca0-cac6-4d2b-9ef4-fe1ea1578a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:47.792740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a220ca0-cac6-4d2b-9ef4-fe1ea1578a2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:47.793608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a220ca0-cac6-4d2b-9ef4-fe1ea1578a2f None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.domain.id=default, user.id=52894fe1eaba4d88aa400c34b4d5791b) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:47.794498 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a220ca0-cac6-4d2b-9ef4-fe1ea1578a2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:47.795983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a220ca0-cac6-4d2b-9ef4-fe1ea1578a2f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:47.802180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 27/57] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Jun 20 10:50:47 2022] GET /identity/v3/role_assignments?user.id=52894fe1eaba4d88aa400c34b4d5791b&scope.domain.id=default&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:48.765314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd675a-2214-4b02-8686-36aa2201a1b3 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:48.766195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd675a-2214-4b02-8686-36aa2201a1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:48.766314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd675a-2214-4b02-8686-36aa2201a1b3 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:48.768527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 31/58] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:48.777205 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a22f-a9fc-43b2-99b1-9f9f36bfeab3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:48.777602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a22f-a9fc-43b2-99b1-9f9f36bfeab3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:48.777849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17a22f-a9fc-43b2-99b1-9f9f36bfeab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:48.803345 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7f17a22f-a9fc-43b2-99b1-9f9f36bfeab3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:48.804912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7f17a22f-a9fc-43b2-99b1-9f9f36bfeab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:48.876777 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 28/59] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:48.901567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd675a-2214-4b02-8686-36aa2201a1b3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:48.911622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:49.016447 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd675a-2214-4b02-8686-36aa2201a1b3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:49.054356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126075e1-8170-46fb-88f5-d711366da9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:49.055158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126075e1-8170-46fb-88f5-d711366da9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:49.055440 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126075e1-8170-46fb-88f5-d711366da9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:49.055713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126075e1-8170-46fb-88f5-d711366da9f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:49.056943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126075e1-8170-46fb-88f5-d711366da9f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:49.058444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126075e1-8170-46fb-88f5-d711366da9f4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:49.060261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 32/60] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:50:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:49.066779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17a22f-a9fc-43b2-99b1-9f9f36bfeab3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:49.121123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:49.174174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17a22f-a9fc-43b2-99b1-9f9f36bfeab3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:49.209857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59091a71-a85f-4c63-9c85-3f29b68f0f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:49.210558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59091a71-a85f-4c63-9c85-3f29b68f0f64 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:49.210850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59091a71-a85f-4c63-9c85-3f29b68f0f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:49.211137 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59091a71-a85f-4c63-9c85-3f29b68f0f64 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:49.211877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59091a71-a85f-4c63-9c85-3f29b68f0f64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:49.213361 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59091a71-a85f-4c63-9c85-3f29b68f0f64 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:49.241099 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 29/61] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:50:49 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:50:52.506470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4174eb-9371-4599-95fd-25f6dcd54ab1 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:52.507519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4174eb-9371-4599-95fd-25f6dcd54ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:52.508004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4174eb-9371-4599-95fd-25f6dcd54ab1 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:52.509896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 33/62] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:52.515641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82af59-22e0-40f9-8189-e60b5928bbbd None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:52.516061 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82af59-22e0-40f9-8189-e60b5928bbbd None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:52.516208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82af59-22e0-40f9-8189-e60b5928bbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:52.535862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7e82af59-22e0-40f9-8189-e60b5928bbbd None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:52.537271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7e82af59-22e0-40f9-8189-e60b5928bbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:52.597043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76e25e-11be-4254-9675-214a26703ff4 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:52.597379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76e25e-11be-4254-9675-214a26703ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:52.597614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76e25e-11be-4254-9675-214a26703ff4 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:52.599050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 34/63] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:50:52.613728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2363a4a-8624-4931-8fbd-56a4caac6d82 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:52.614031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2363a4a-8624-4931-8fbd-56a4caac6d82 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:52.614323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2363a4a-8624-4931-8fbd-56a4caac6d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:52.641768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c2363a4a-8624-4931-8fbd-56a4caac6d82 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:52.643113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c2363a4a-8624-4931-8fbd-56a4caac6d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:52.645199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 30/64] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:52.646897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b5a83-3680-421f-b98c-1ee37ac2a621 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:52.647402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b5a83-3680-421f-b98c-1ee37ac2a621 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:52.647894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b5a83-3680-421f-b98c-1ee37ac2a621 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:52.649423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 31/65] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:52.654817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cc640-c614-41d1-b3bd-4e520e240c4b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:52.655347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cc640-c614-41d1-b3bd-4e520e240c4b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:52.655818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cc640-c614-41d1-b3bd-4e520e240c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:52.683542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a66cc640-c614-41d1-b3bd-4e520e240c4b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:52.685979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a66cc640-c614-41d1-b3bd-4e520e240c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:52.723960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:52.733251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 35/66] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:52.734514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2363a4a-8624-4931-8fbd-56a4caac6d82 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:52.740577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:52.785351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 32/67] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:50:52.792029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cc640-c614-41d1-b3bd-4e520e240c4b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:52.865438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2363a4a-8624-4931-8fbd-56a4caac6d82 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:52.901266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cc640-c614-41d1-b3bd-4e520e240c4b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:52.902263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b83064-165e-48a4-9624-479d0cd7d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:52.903766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b83064-165e-48a4-9624-479d0cd7d755 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:52.905087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b83064-165e-48a4-9624-479d0cd7d755 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:52.905504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b83064-165e-48a4-9624-479d0cd7d755 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:52.921034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b83064-165e-48a4-9624-479d0cd7d755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d1b83064-165e-48a4-9624-479d0cd7d755 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:52.929683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:52.930584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:50:52.931327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 36/68] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:50:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:52.932594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6676ec-52e4-4f85-93b8-705371b747ed None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:52.932594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6676ec-52e4-4f85-93b8-705371b747ed None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:52.932743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6676ec-52e4-4f85-93b8-705371b747ed None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:52.938206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 37/69] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:52.938206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6676ec-52e4-4f85-93b8-705371b747ed None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:52.943635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84720b-92e4-4352-914c-771bef322ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:52.944447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84720b-92e4-4352-914c-771bef322ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:52.944730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84720b-92e4-4352-914c-771bef322ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:52.945009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84720b-92e4-4352-914c-771bef322ee1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:52.956369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84720b-92e4-4352-914c-771bef322ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-fb84720b-92e4-4352-914c-771bef322ee1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:52.964439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:52.965673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:50:52.966724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:50:52.967282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 33/70] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:50:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:52.967622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84720b-92e4-4352-914c-771bef322ee1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.001229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:53.016204 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:53.058811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6676ec-52e4-4f85-93b8-705371b747ed None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:53.083454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84720b-92e4-4352-914c-771bef322ee1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:53.101973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f6219-01b3-4209-9b6a-5e730f28ed92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:53.102929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f6219-01b3-4209-9b6a-5e730f28ed92 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.103209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f6219-01b3-4209-9b6a-5e730f28ed92 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.103481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f6219-01b3-4209-9b6a-5e730f28ed92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.116814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f6219-01b3-4209-9b6a-5e730f28ed92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:53.121993 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816504c-f769-45af-a803-09a0a77bdeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:53.122710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816504c-f769-45af-a803-09a0a77bdeba None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.123004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816504c-f769-45af-a803-09a0a77bdeba None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.123281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816504c-f769-45af-a803-09a0a77bdeba None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.124051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816504c-f769-45af-a803-09a0a77bdeba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:53.124370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816504c-f769-45af-a803-09a0a77bdeba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-613f6219-01b3-4209-9b6a-5e730f28ed92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:53.125382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:53.126371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:50:53.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:50:53.127809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816504c-f769-45af-a803-09a0a77bdeba None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:53.127873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 38/71] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:50:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:53.127873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e47be-4007-4471-9801-cf7cab4c3dac None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.128427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e47be-4007-4471-9801-cf7cab4c3dac None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.128427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e47be-4007-4471-9801-cf7cab4c3dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.143738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 34/72] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:50:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:53.144596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816504c-f769-45af-a803-09a0a77bdeba None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.145369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6c8e47be-4007-4471-9801-cf7cab4c3dac None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:53.146581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6c8e47be-4007-4471-9801-cf7cab4c3dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:53.212808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 39/73] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:53.213939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b3283-3fde-4c93-b5f2-3a871becfc4c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.214261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b3283-3fde-4c93-b5f2-3a871becfc4c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.214538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b3283-3fde-4c93-b5f2-3a871becfc4c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.215895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 40/74] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:53.216806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b3283-3fde-4c93-b5f2-3a871becfc4c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.242042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:53.256881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816504c-f769-45af-a803-09a0a77bdeba None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:53.271764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:53.290288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1fa2f-580a-45d9-a3e5-30e1fee98ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:53.290971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1fa2f-580a-45d9-a3e5-30e1fee98ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.291256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1fa2f-580a-45d9-a3e5-30e1fee98ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.291539 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1fa2f-580a-45d9-a3e5-30e1fee98ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.292448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1fa2f-580a-45d9-a3e5-30e1fee98ea2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:53.292590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1fa2f-580a-45d9-a3e5-30e1fee98ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:53.294176 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1fa2f-580a-45d9-a3e5-30e1fee98ea2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:53.302286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 35/75] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:50:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:53.303077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1fa2f-580a-45d9-a3e5-30e1fee98ea2 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.319358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b3283-3fde-4c93-b5f2-3a871becfc4c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:53.354440 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd7958-d30f-4e90-8a4b-c989b68d31c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:53.355113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd7958-d30f-4e90-8a4b-c989b68d31c1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.355400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd7958-d30f-4e90-8a4b-c989b68d31c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.355680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd7958-d30f-4e90-8a4b-c989b68d31c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.356528 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd7958-d30f-4e90-8a4b-c989b68d31c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:53.356832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd7958-d30f-4e90-8a4b-c989b68d31c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:53.358257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd7958-d30f-4e90-8a4b-c989b68d31c1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:53.366076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 41/76] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:50:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:53.367342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76cd6-d0d6-4490-a498-57474fdb4f9f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.367639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76cd6-d0d6-4490-a498-57474fdb4f9f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.367931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd76cd6-d0d6-4490-a498-57474fdb4f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.383610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0fd76cd6-d0d6-4490-a498-57474fdb4f9f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:53.384808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0fd76cd6-d0d6-4490-a498-57474fdb4f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:53.405201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1fa2f-580a-45d9-a3e5-30e1fee98ea2 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:53.439398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35484e2a-4d8c-4c96-897e-5f31d24a1682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:53.440293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35484e2a-4d8c-4c96-897e-5f31d24a1682 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.440560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35484e2a-4d8c-4c96-897e-5f31d24a1682 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.440832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35484e2a-4d8c-4c96-897e-5f31d24a1682 None admin] PATH_INFO: `/v3/users/52894fe1eaba4d88aa400c34b4d5791b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.442036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35484e2a-4d8c-4c96-897e-5f31d24a1682 None admin] RBAC: Authorizing `identity:get_user(user_id=52894fe1eaba4d88aa400c34b4d5791b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:53.443536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35484e2a-4d8c-4c96-897e-5f31d24a1682 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:53.445391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 36/77] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:50:53 2022] GET /identity/v3/users/52894fe1eaba4d88aa400c34b4d5791b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:53.445959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35484e2a-4d8c-4c96-897e-5f31d24a1682 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.460068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 42/78] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:53.460939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76cd6-d0d6-4490-a498-57474fdb4f9f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.471906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:53.518605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:53.555969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35484e2a-4d8c-4c96-897e-5f31d24a1682 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:53.572293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd76cd6-d0d6-4490-a498-57474fdb4f9f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:53.590192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f516f-9794-47bd-a6ae-284398fb6df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:53.591142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f516f-9794-47bd-a6ae-284398fb6df7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.591425 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f516f-9794-47bd-a6ae-284398fb6df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.591702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f516f-9794-47bd-a6ae-284398fb6df7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.601749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f516f-9794-47bd-a6ae-284398fb6df7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:53.607750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc9815-5ed8-4e26-9c8e-698a40aa685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:53.608643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc9815-5ed8-4e26-9c8e-698a40aa685b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.608923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc9815-5ed8-4e26-9c8e-698a40aa685b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.609199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc9815-5ed8-4e26-9c8e-698a40aa685b None admin] PATH_INFO: `/v3/users/6fdb844dddc042b9958d0532be8d63c7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0c6f516f-9794-47bd-a6ae-284398fb6df7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:53.609552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:53.610349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:50:53.611104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 37/79] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:50:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:53.611728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f516f-9794-47bd-a6ae-284398fb6df7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.627501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc9815-5ed8-4e26-9c8e-698a40aa685b None admin] RBAC: Authorizing `identity:get_user(user_id=6fdb844dddc042b9958d0532be8d63c7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:53.628991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc9815-5ed8-4e26-9c8e-698a40aa685b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:53.630730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 43/80] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:50:53 2022] GET /identity/v3/users/6fdb844dddc042b9958d0532be8d63c7 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:53.634396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc9815-5ed8-4e26-9c8e-698a40aa685b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.687576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:53.711098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f516f-9794-47bd-a6ae-284398fb6df7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:53.732532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc9815-5ed8-4e26-9c8e-698a40aa685b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:53.734905 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:53.744380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d740f-4726-4f1a-9794-f1eb135ce378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:53.745267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d740f-4726-4f1a-9794-f1eb135ce378 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.745552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d740f-4726-4f1a-9794-f1eb135ce378 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.745827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d740f-4726-4f1a-9794-f1eb135ce378 None admin] PATH_INFO: `/v3/users/6fdb844dddc042b9958d0532be8d63c7` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.747268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d740f-4726-4f1a-9794-f1eb135ce378 None admin] RBAC: Authorizing `identity:get_user(user_id=6fdb844dddc042b9958d0532be8d63c7)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:53.748757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d740f-4726-4f1a-9794-f1eb135ce378 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:53.750465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 38/81] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:50:53 2022] GET /identity/v3/users/6fdb844dddc042b9958d0532be8d63c7 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:53.751222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d740f-4726-4f1a-9794-f1eb135ce378 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.770175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8242cd-d279-4791-997a-ad23fc9be9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:53.770825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8242cd-d279-4791-997a-ad23fc9be9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.771109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8242cd-d279-4791-997a-ad23fc9be9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.771379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8242cd-d279-4791-997a-ad23fc9be9c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.839199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8242cd-d279-4791-997a-ad23fc9be9c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:53.840730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8242cd-d279-4791-997a-ad23fc9be9c0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:53.848896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d740f-4726-4f1a-9794-f1eb135ce378 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:53.884091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0004c-28a3-4bf1-bf18-e213646f7096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:53.884904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0004c-28a3-4bf1-bf18-e213646f7096 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:53.885190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0004c-28a3-4bf1-bf18-e213646f7096 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:53.885473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0004c-28a3-4bf1-bf18-e213646f7096 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:53.886712 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0004c-28a3-4bf1-bf18-e213646f7096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:53.888203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0004c-28a3-4bf1-bf18-e213646f7096 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:53.889856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 39/82] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:50:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:53.890660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0004c-28a3-4bf1-bf18-e213646f7096 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.910275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:53.943088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 44/83] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:50:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:50:53.943945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8242cd-d279-4791-997a-ad23fc9be9c0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:53.947723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:54.004994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0004c-28a3-4bf1-bf18-e213646f7096 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:54.038235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdd274-e140-4dae-bb79-619163035561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:54.039008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdd274-e140-4dae-bb79-619163035561 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:54.039388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdd274-e140-4dae-bb79-619163035561 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:54.039690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdd274-e140-4dae-bb79-619163035561 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:54.040481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdd274-e140-4dae-bb79-619163035561 None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:54.040798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdd274-e140-4dae-bb79-619163035561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:54.042133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8242cd-d279-4791-997a-ad23fc9be9c0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:54.042235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdd274-e140-4dae-bb79-619163035561 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:54.050513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 40/84] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:50:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:54.051349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdd274-e140-4dae-bb79-619163035561 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:54.076547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92930015-d3b3-41c7-b095-6367a3bd6627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:54.077199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92930015-d3b3-41c7-b095-6367a3bd6627 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:54.077510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92930015-d3b3-41c7-b095-6367a3bd6627 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:54.077780 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92930015-d3b3-41c7-b095-6367a3bd6627 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:54.125033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:54.146038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92930015-d3b3-41c7-b095-6367a3bd6627 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:54.147551 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92930015-d3b3-41c7-b095-6367a3bd6627 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:54.149752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdd274-e140-4dae-bb79-619163035561 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:54.150891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:54.182848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847e344-6450-4beb-97a1-bee4a3c89c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:54.183501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847e344-6450-4beb-97a1-bee4a3c89c76 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:54.183793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847e344-6450-4beb-97a1-bee4a3c89c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:54.184120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847e344-6450-4beb-97a1-bee4a3c89c76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:54.244830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 45/85] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:50:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:50:54.245638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92930015-d3b3-41c7-b095-6367a3bd6627 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:54.253148 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847e344-6450-4beb-97a1-bee4a3c89c76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:54.254703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847e344-6450-4beb-97a1-bee4a3c89c76 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:54.327150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:54.343028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92930015-d3b3-41c7-b095-6367a3bd6627 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:54.345574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:54.353240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 41/86] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:50:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:50:54.354075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847e344-6450-4beb-97a1-bee4a3c89c76 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:54.377772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae0151-2835-47bc-bf5d-ff303230e14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:54.378411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae0151-2835-47bc-bf5d-ff303230e14b None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:54.378691 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae0151-2835-47bc-bf5d-ff303230e14b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:54.378964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae0151-2835-47bc-bf5d-ff303230e14b None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:54.379684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae0151-2835-47bc-bf5d-ff303230e14b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:54.381159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae0151-2835-47bc-bf5d-ff303230e14b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:54.407631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 46/87] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:50:54 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:50:54.408486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae0151-2835-47bc-bf5d-ff303230e14b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:54.488585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847e344-6450-4beb-97a1-bee4a3c89c76 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:54.529947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae0151-2835-47bc-bf5d-ff303230e14b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:54.531885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18750c-64a2-45d3-aafe-b8485a608e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:54.532949 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18750c-64a2-45d3-aafe-b8485a608e42 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:54.533240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18750c-64a2-45d3-aafe-b8485a608e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:54.533526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18750c-64a2-45d3-aafe-b8485a608e42 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:54.534821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18750c-64a2-45d3-aafe-b8485a608e42 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:54.536326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18750c-64a2-45d3-aafe-b8485a608e42 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:54.543177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 42/88] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:50:54 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:54.543177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18750c-64a2-45d3-aafe-b8485a608e42 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:54.577795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46a5d6-4b4e-4abc-84b3-877fd1fbafd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:54.578726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46a5d6-4b4e-4abc-84b3-877fd1fbafd1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:54.579009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46a5d6-4b4e-4abc-84b3-877fd1fbafd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:54.579287 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46a5d6-4b4e-4abc-84b3-877fd1fbafd1 None admin] PATH_INFO: `/v3/users/6fdb844dddc042b9958d0532be8d63c7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:54.581472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46a5d6-4b4e-4abc-84b3-877fd1fbafd1 None admin] RBAC: Authorizing `identity:get_user(user_id=6fdb844dddc042b9958d0532be8d63c7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:54.582968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46a5d6-4b4e-4abc-84b3-877fd1fbafd1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:54.584919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 47/89] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:50:54 2022] GET /identity/v3/users/6fdb844dddc042b9958d0532be8d63c7 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:54.590709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46a5d6-4b4e-4abc-84b3-877fd1fbafd1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:54.592106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:54.623286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:54.663142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18750c-64a2-45d3-aafe-b8485a608e42 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:54.709661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6dc12-92bb-404c-ae4d-bf6d2a77492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:54.711294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6dc12-92bb-404c-ae4d-bf6d2a77492f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:54.712001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6dc12-92bb-404c-ae4d-bf6d2a77492f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:54.712447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6dc12-92bb-404c-ae4d-bf6d2a77492f None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:54.713602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46a5d6-4b4e-4abc-84b3-877fd1fbafd1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:54.714036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6dc12-92bb-404c-ae4d-bf6d2a77492f None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:54.716082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6dc12-92bb-404c-ae4d-bf6d2a77492f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:54.719024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 43/90] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:50:54 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:54.720010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6dc12-92bb-404c-ae4d-bf6d2a77492f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:54.756660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7fcd9-6c76-45b3-92de-fe0d32561c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:54.757748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7fcd9-6c76-45b3-92de-fe0d32561c04 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:54.758039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7fcd9-6c76-45b3-92de-fe0d32561c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:54.758716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7fcd9-6c76-45b3-92de-fe0d32561c04 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:54.760214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7fcd9-6c76-45b3-92de-fe0d32561c04 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:54.762902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7fcd9-6c76-45b3-92de-fe0d32561c04 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:54.764894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 48/91] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:50:54 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:54.765478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7fcd9-6c76-45b3-92de-fe0d32561c04 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:54.856241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6dc12-92bb-404c-ae4d-bf6d2a77492f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:54.882513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:54.887088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:54.895451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7fcd9-6c76-45b3-92de-fe0d32561c04 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:54.901346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21bb2e-18a2-4b5c-b689-3e30e12e9af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:54.902027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21bb2e-18a2-4b5c-b689-3e30e12e9af1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:54.902345 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21bb2e-18a2-4b5c-b689-3e30e12e9af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:54.902627 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21bb2e-18a2-4b5c-b689-3e30e12e9af1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:54.906124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21bb2e-18a2-4b5c-b689-3e30e12e9af1 None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=9dd8fde445f5496fab61c82990f8ba09, user.id=52894fe1eaba4d88aa400c34b4d5791b) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:54.906124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21bb2e-18a2-4b5c-b689-3e30e12e9af1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:54.906124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21bb2e-18a2-4b5c-b689-3e30e12e9af1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:54.914904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 44/92] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:50:54 2022] GET /identity/v3/role_assignments?user.id=52894fe1eaba4d88aa400c34b4d5791b&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:54.915048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21bb2e-18a2-4b5c-b689-3e30e12e9af1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:54.961310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bc23a-86eb-4a47-8142-762f7afd050f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:54.961989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bc23a-86eb-4a47-8142-762f7afd050f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:54.962291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bc23a-86eb-4a47-8142-762f7afd050f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:54.962566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bc23a-86eb-4a47-8142-762f7afd050f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:55.093785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bc23a-86eb-4a47-8142-762f7afd050f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:55.095650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bc23a-86eb-4a47-8142-762f7afd050f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:55.117912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21bb2e-18a2-4b5c-b689-3e30e12e9af1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:55.188031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048ae37-07f5-4900-aafc-408e6254ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:55.188617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048ae37-07f5-4900-aafc-408e6254ad22 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:55.188899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048ae37-07f5-4900-aafc-408e6254ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:55.189175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048ae37-07f5-4900-aafc-408e6254ad22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:55.190022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048ae37-07f5-4900-aafc-408e6254ad22 None admin] RBAC: Adding query filter params (role.id=3e46d43e30ef494087da2c44f7ca702f, scope.project.id=9dd8fde445f5496fab61c82990f8ba09, user.id=6fdb844dddc042b9958d0532be8d63c7) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:55.190370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048ae37-07f5-4900-aafc-408e6254ad22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:55.191853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048ae37-07f5-4900-aafc-408e6254ad22 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:55.204675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 45/93] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:50:54 2022] GET /identity/v3/role_assignments?user.id=6fdb844dddc042b9958d0532be8d63c7&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:55.205289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048ae37-07f5-4900-aafc-408e6254ad22 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:55.263291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:55.272106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 49/94] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:50:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:50:55.279200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bc23a-86eb-4a47-8142-762f7afd050f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:55.326561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:55.400227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bc23a-86eb-4a47-8142-762f7afd050f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:55.440581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5a2b5-d61f-4eb2-ad43-4ea05aeffad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:55.441385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5a2b5-d61f-4eb2-ad43-4ea05aeffad8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:55.441664 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5a2b5-d61f-4eb2-ad43-4ea05aeffad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:55.441935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5a2b5-d61f-4eb2-ad43-4ea05aeffad8 None admin] PATH_INFO: `/v3/projects/73b2d8f446f84ec69cdd45be83d60f38` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:55.443288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5a2b5-d61f-4eb2-ad43-4ea05aeffad8 None admin] RBAC: Authorizing `identity:get_project(project_id=73b2d8f446f84ec69cdd45be83d60f38)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:55.444859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5a2b5-d61f-4eb2-ad43-4ea05aeffad8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:55.454402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 50/95] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:50:55 2022] GET /identity/v3/projects/73b2d8f446f84ec69cdd45be83d60f38 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:55.455317 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5a2b5-d61f-4eb2-ad43-4ea05aeffad8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:55.468039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048ae37-07f5-4900-aafc-408e6254ad22 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:55.504204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f63534-b635-4a03-b226-76d586cc629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:55.504872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f63534-b635-4a03-b226-76d586cc629b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:55.505157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f63534-b635-4a03-b226-76d586cc629b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:55.505435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f63534-b635-4a03-b226-76d586cc629b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:55.506315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f63534-b635-4a03-b226-76d586cc629b None admin] RBAC: Adding query filter params (role.id=ffab93457c0f4f64a29764f17759396e, scope.project.id=9dd8fde445f5496fab61c82990f8ba09, user.id=6fdb844dddc042b9958d0532be8d63c7) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:55.506634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f63534-b635-4a03-b226-76d586cc629b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:55.508105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f63534-b635-4a03-b226-76d586cc629b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:55.514517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 46/96] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:50:55 2022] GET /identity/v3/role_assignments?user.id=6fdb844dddc042b9958d0532be8d63c7&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=ffab93457c0f4f64a29764f17759396e => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:55.561661 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:55.619852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5a2b5-d61f-4eb2-ad43-4ea05aeffad8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:55.659635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5cdeb-1e2d-483a-8731-59e7260b4c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:55.660311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5cdeb-1e2d-483a-8731-59e7260b4c91 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:55.660590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5cdeb-1e2d-483a-8731-59e7260b4c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:55.660863 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5cdeb-1e2d-483a-8731-59e7260b4c91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:55.661719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5cdeb-1e2d-483a-8731-59e7260b4c91 None admin] RBAC: Adding query filter params (role.id=3e46d43e30ef494087da2c44f7ca702f, scope.project.id=73b2d8f446f84ec69cdd45be83d60f38, user.id=6fdb844dddc042b9958d0532be8d63c7) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:55.662031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5cdeb-1e2d-483a-8731-59e7260b4c91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:55.663570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5cdeb-1e2d-483a-8731-59e7260b4c91 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:55.673220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 51/97] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:50:55 2022] GET /identity/v3/role_assignments?user.id=6fdb844dddc042b9958d0532be8d63c7&scope.project.id=73b2d8f446f84ec69cdd45be83d60f38&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:57.084657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19eab7-3924-4dde-bd0b-488d20749090 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:57.085670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19eab7-3924-4dde-bd0b-488d20749090 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:57.086193 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19eab7-3924-4dde-bd0b-488d20749090 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:57.087808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 47/98] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:57.093801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432f1b0-a167-49d9-8ba9-162f99cdbaf3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:57.094701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432f1b0-a167-49d9-8ba9-162f99cdbaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:57.095170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432f1b0-a167-49d9-8ba9-162f99cdbaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:57.117820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9432f1b0-a167-49d9-8ba9-162f99cdbaf3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:57.119405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9432f1b0-a167-49d9-8ba9-162f99cdbaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:57.209758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 52/99] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:57.233763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19eab7-3924-4dde-bd0b-488d20749090 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:57.366393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19eab7-3924-4dde-bd0b-488d20749090 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:57.367928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:57.414789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd8068-88de-4a6c-b27d-a9a6dd983644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:57.415601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd8068-88de-4a6c-b27d-a9a6dd983644 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:57.415883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd8068-88de-4a6c-b27d-a9a6dd983644 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:57.416200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd8068-88de-4a6c-b27d-a9a6dd983644 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:57.435489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd8068-88de-4a6c-b27d-a9a6dd983644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-c8fd8068-88de-4a6c-b27d-a9a6dd983644 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:57.443442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:57.444320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:50:57.445054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 48/100] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:50:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:57.451099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432f1b0-a167-49d9-8ba9-162f99cdbaf3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:57.513600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:57.559558 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432f1b0-a167-49d9-8ba9-162f99cdbaf3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:57.595961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e69a84-ffb5-4631-b418-0f9548b45e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:57.596641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e69a84-ffb5-4631-b418-0f9548b45e73 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:57.596923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e69a84-ffb5-4631-b418-0f9548b45e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:57.597199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e69a84-ffb5-4631-b418-0f9548b45e73 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:57.597955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e69a84-ffb5-4631-b418-0f9548b45e73 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:57.598293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e69a84-ffb5-4631-b418-0f9548b45e73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:57.599685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e69a84-ffb5-4631-b418-0f9548b45e73 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:57.609200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 53/101] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:50:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:57.614898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd8068-88de-4a6c-b27d-a9a6dd983644 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:57.690129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eb7fd-2547-457b-be0b-55a9f0d0b3a5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:57.690129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eb7fd-2547-457b-be0b-55a9f0d0b3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:57.690129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eb7fd-2547-457b-be0b-55a9f0d0b3a5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:57.691714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 54/102] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:57.696948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34915ac-0f67-456a-a2cc-d072682ded81 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:57.697446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34915ac-0f67-456a-a2cc-d072682ded81 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:57.697910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34915ac-0f67-456a-a2cc-d072682ded81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:57.727877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c34915ac-0f67-456a-a2cc-d072682ded81 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:57.729093 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c34915ac-0f67-456a-a2cc-d072682ded81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:57.780878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd8068-88de-4a6c-b27d-a9a6dd983644 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:57.844803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1a449-cdcf-48f7-98ab-b5b580409c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:57.845702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1a449-cdcf-48f7-98ab-b5b580409c15 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:57.845986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1a449-cdcf-48f7-98ab-b5b580409c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:57.846423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1a449-cdcf-48f7-98ab-b5b580409c15 None admin] PATH_INFO: `/v3/users/c4793dfbf0514b038e9957c59546b023` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:57.854126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 55/103] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:57.856142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9841673-8ad7-46eb-bcbb-2ac0e32457d5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:57.856633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9841673-8ad7-46eb-bcbb-2ac0e32457d5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:57.857117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9841673-8ad7-46eb-bcbb-2ac0e32457d5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:57.863730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 56/104] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:57.865476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c5472-95de-47d4-b8de-2783d918735b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:57.865951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c5472-95de-47d4-b8de-2783d918735b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:57.866436 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c5472-95de-47d4-b8de-2783d918735b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:57.871873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1a449-cdcf-48f7-98ab-b5b580409c15 None admin] RBAC: Authorizing `identity:get_user(user_id=c4793dfbf0514b038e9957c59546b023)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:57.873691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1a449-cdcf-48f7-98ab-b5b580409c15 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:57.875741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 49/105] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:50:57 2022] GET /identity/v3/users/c4793dfbf0514b038e9957c59546b023 => generated 289 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:50:57.882115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1a449-cdcf-48f7-98ab-b5b580409c15 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:57.898487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-775c5472-95de-47d4-b8de-2783d918735b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:57.900352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-775c5472-95de-47d4-b8de-2783d918735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:57.921567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:57.994445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:58.018208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 57/106] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:58.019217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c5472-95de-47d4-b8de-2783d918735b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:58.079335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1a449-cdcf-48f7-98ab-b5b580409c15 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:58.114220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62ebb0-aae2-41ba-b90d-79cd0cee87c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:58.114990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62ebb0-aae2-41ba-b90d-79cd0cee87c7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:58.115218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62ebb0-aae2-41ba-b90d-79cd0cee87c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:58.115558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62ebb0-aae2-41ba-b90d-79cd0cee87c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:58.186910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad62ebb0-aae2-41ba-b90d-79cd0cee87c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:58.188501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad62ebb0-aae2-41ba-b90d-79cd0cee87c7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:58.210514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:58.213369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c5472-95de-47d4-b8de-2783d918735b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:58.273373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e880a-c455-4fbe-94f0-671ab8b6b915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:58.274244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e880a-c455-4fbe-94f0-671ab8b6b915 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:58.274530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e880a-c455-4fbe-94f0-671ab8b6b915 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:58.275091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e880a-c455-4fbe-94f0-671ab8b6b915 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:58.289085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e880a-c455-4fbe-94f0-671ab8b6b915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:58.298360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 50/107] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:50:58.299339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62ebb0-aae2-41ba-b90d-79cd0cee87c7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-993e880a-c455-4fbe-94f0-671ab8b6b915 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:58.299472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:58.300260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:50:58.301207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 58/108] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:50:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:58.301873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0621b0b-7c39-4f40-b039-d4b45d48df0a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:58.303538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0621b0b-7c39-4f40-b039-d4b45d48df0a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:58.303538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0621b0b-7c39-4f40-b039-d4b45d48df0a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:58.303982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 59/109] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:58.309864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0621b0b-7c39-4f40-b039-d4b45d48df0a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:58.386654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:58.430268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62ebb0-aae2-41ba-b90d-79cd0cee87c7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:58.447304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0621b0b-7c39-4f40-b039-d4b45d48df0a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:58.455896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:58.474675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36092ccd-f796-467d-a06d-4203b019a0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:58.475642 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36092ccd-f796-467d-a06d-4203b019a0ca None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:58.476030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36092ccd-f796-467d-a06d-4203b019a0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:58.476882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36092ccd-f796-467d-a06d-4203b019a0ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:58.478487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36092ccd-f796-467d-a06d-4203b019a0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:58.479988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36092ccd-f796-467d-a06d-4203b019a0ca None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:58.482372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 51/110] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:50:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:58.482919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36092ccd-f796-467d-a06d-4203b019a0ca None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:58.502244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59fc89-2bdd-419a-920a-3288f1cd2668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:58.503486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59fc89-2bdd-419a-920a-3288f1cd2668 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:58.504187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59fc89-2bdd-419a-920a-3288f1cd2668 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:58.504589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59fc89-2bdd-419a-920a-3288f1cd2668 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:58.506075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59fc89-2bdd-419a-920a-3288f1cd2668 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:58.509420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59fc89-2bdd-419a-920a-3288f1cd2668 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:58.513602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 60/111] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:50:58 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:58.519512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac6764-d492-40e6-97d3-6561fed1ee6a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:58.519798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac6764-d492-40e6-97d3-6561fed1ee6a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:58.520074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac6764-d492-40e6-97d3-6561fed1ee6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:58.546649 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e3ac6764-d492-40e6-97d3-6561fed1ee6a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:58.548259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e3ac6764-d492-40e6-97d3-6561fed1ee6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:58.605120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36092ccd-f796-467d-a06d-4203b019a0ca None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:58.625788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 61/112] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:58.627003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac6764-d492-40e6-97d3-6561fed1ee6a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:58.644491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf064c5d-fa19-409d-9b56-56e5f6b5c910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:58.645307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf064c5d-fa19-409d-9b56-56e5f6b5c910 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:58.645695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf064c5d-fa19-409d-9b56-56e5f6b5c910 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:58.646073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf064c5d-fa19-409d-9b56-56e5f6b5c910 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:58.646959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf064c5d-fa19-409d-9b56-56e5f6b5c910 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:58.648042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf064c5d-fa19-409d-9b56-56e5f6b5c910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:58.649480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf064c5d-fa19-409d-9b56-56e5f6b5c910 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:58.659175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 52/113] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:50:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:58.660014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf064c5d-fa19-409d-9b56-56e5f6b5c910 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:58.685288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:58.716672 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:58.745831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac6764-d492-40e6-97d3-6561fed1ee6a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:58.768150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf064c5d-fa19-409d-9b56-56e5f6b5c910 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:58.789909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6433d-076c-4b93-b41f-f11af39403a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:58.790614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6433d-076c-4b93-b41f-f11af39403a6 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:58.790997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6433d-076c-4b93-b41f-f11af39403a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:58.791279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6433d-076c-4b93-b41f-f11af39403a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:58.792001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6433d-076c-4b93-b41f-f11af39403a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:58.793587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6433d-076c-4b93-b41f-f11af39403a6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:58.802520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a0643-184c-4cf4-990c-baee4365433e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:58.803187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a0643-184c-4cf4-990c-baee4365433e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:58.803471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a0643-184c-4cf4-990c-baee4365433e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:58.804001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a0643-184c-4cf4-990c-baee4365433e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:58.804590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a0643-184c-4cf4-990c-baee4365433e None admin] RBAC: Adding query filter params (role.id=e4214ef8456541528194f90e229b356a, scope.project.id=9dd8fde445f5496fab61c82990f8ba09, user.id=c4793dfbf0514b038e9957c59546b023) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:58.804905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a0643-184c-4cf4-990c-baee4365433e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:58.806550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a0643-184c-4cf4-990c-baee4365433e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:58.813160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 53/114] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:50:58 2022] GET /identity/v3/role_assignments?user.id=c4793dfbf0514b038e9957c59546b023&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=e4214ef8456541528194f90e229b356a => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:58.825606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace13a-9700-4856-8034-0ec8e1fd33ef None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:58.825606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace13a-9700-4856-8034-0ec8e1fd33ef None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:58.825606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace13a-9700-4856-8034-0ec8e1fd33ef None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:58.825606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 54/115] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:58.825606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace13a-9700-4856-8034-0ec8e1fd33ef None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:58.830365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 62/116] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:50:58 2022] POST /identity/v3/projects => generated 302 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:50:58.831820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6433d-076c-4b93-b41f-f11af39403a6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:58.950654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:58.964274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6433d-076c-4b93-b41f-f11af39403a6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:58.966781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace13a-9700-4856-8034-0ec8e1fd33ef None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:58.984255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:59.010315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa8ae9-2331-4217-8b8e-5c1a0724071c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:59.011041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c25d1b-aa31-48ba-92fe-d61c7a354d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:59.011852 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa8ae9-2331-4217-8b8e-5c1a0724071c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.011980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c25d1b-aa31-48ba-92fe-d61c7a354d6b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.012292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c25d1b-aa31-48ba-92fe-d61c7a354d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.012575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c25d1b-aa31-48ba-92fe-d61c7a354d6b None admin] PATH_INFO: `/v3/users/52894fe1eaba4d88aa400c34b4d5791b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:59.012931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa8ae9-2331-4217-8b8e-5c1a0724071c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.013401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa8ae9-2331-4217-8b8e-5c1a0724071c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:59.013961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c25d1b-aa31-48ba-92fe-d61c7a354d6b None admin] RBAC: Authorizing `identity:get_user(user_id=52894fe1eaba4d88aa400c34b4d5791b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:59.015547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c25d1b-aa31-48ba-92fe-d61c7a354d6b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:59.017665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 63/117] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:50:58 2022] GET /identity/v3/users/52894fe1eaba4d88aa400c34b4d5791b => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:59.018938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00090fc3-07a4-4955-ab8c-2c4203230f5b None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.019231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00090fc3-07a4-4955-ab8c-2c4203230f5b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.019512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00090fc3-07a4-4955-ab8c-2c4203230f5b None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:59.020827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 64/118] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:50:59.026660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdf0cb-d8ad-4cb9-b7a3-bec84ffcbda9 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.026956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdf0cb-d8ad-4cb9-b7a3-bec84ffcbda9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.027244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdf0cb-d8ad-4cb9-b7a3-bec84ffcbda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:59.028942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa8ae9-2331-4217-8b8e-5c1a0724071c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-98fa8ae9-2331-4217-8b8e-5c1a0724071c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:59.037364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:59.038294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:50:59.039048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 55/119] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:50:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:59.039681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa8ae9-2331-4217-8b8e-5c1a0724071c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:59.044746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-aefdf0cb-d8ad-4cb9-b7a3-bec84ffcbda9 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:59.046491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-aefdf0cb-d8ad-4cb9-b7a3-bec84ffcbda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:59.139698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 65/120] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:59.141550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca2ff-f980-4f61-8b02-40c8c123ff36 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.142269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca2ff-f980-4f61-8b02-40c8c123ff36 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.142721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca2ff-f980-4f61-8b02-40c8c123ff36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:59.163730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ce1ca2ff-f980-4f61-8b02-40c8c123ff36 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:50:59.165501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ce1ca2ff-f980-4f61-8b02-40c8c123ff36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:50:59.169125 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa8ae9-2331-4217-8b8e-5c1a0724071c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:59.222195 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88921a53-e8b4-4408-a35d-2df26c066db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:59.223728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88921a53-e8b4-4408-a35d-2df26c066db4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.224436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88921a53-e8b4-4408-a35d-2df26c066db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.225758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88921a53-e8b4-4408-a35d-2df26c066db4 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:59.229317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88921a53-e8b4-4408-a35d-2df26c066db4 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:59.231342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88921a53-e8b4-4408-a35d-2df26c066db4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:59.234710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 56/121] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:50:59 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:59.241002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88921a53-e8b4-4408-a35d-2df26c066db4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:59.252957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 66/122] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:50:59.253739 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ca2ff-f980-4f61-8b02-40c8c123ff36 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:59.279505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:59.299256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:59.375882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88921a53-e8b4-4408-a35d-2df26c066db4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:59.401762 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ca2ff-f980-4f61-8b02-40c8c123ff36 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:59.411778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916154e7-07de-4597-a6e0-e753a9f4857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:59.412534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916154e7-07de-4597-a6e0-e753a9f4857d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.412919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916154e7-07de-4597-a6e0-e753a9f4857d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.413300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916154e7-07de-4597-a6e0-e753a9f4857d None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:59.414553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916154e7-07de-4597-a6e0-e753a9f4857d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:50:59.414553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916154e7-07de-4597-a6e0-e753a9f4857d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:59.416141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916154e7-07de-4597-a6e0-e753a9f4857d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:59.425656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 57/123] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:50:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:50:59.430876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916154e7-07de-4597-a6e0-e753a9f4857d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:59.449231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef6dd1-fadb-4906-be23-570115d99391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:59.450884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef6dd1-fadb-4906-be23-570115d99391 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.451585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef6dd1-fadb-4906-be23-570115d99391 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.452000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef6dd1-fadb-4906-be23-570115d99391 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:59.466936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef6dd1-fadb-4906-be23-570115d99391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a9ef6dd1-fadb-4906-be23-570115d99391 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:59.477215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:59.478257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:50:59.479219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:50:59.481739 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 67/124] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:50:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:59.483130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef6dd1-fadb-4906-be23-570115d99391 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:59.550193 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916154e7-07de-4597-a6e0-e753a9f4857d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:59.553706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:59.601231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:50:59.606606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252edbc-801e-4904-abf7-cd2a7b6624b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:59.607718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252edbc-801e-4904-abf7-cd2a7b6624b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.608204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252edbc-801e-4904-abf7-cd2a7b6624b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.608594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252edbc-801e-4904-abf7-cd2a7b6624b6 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09/users/52894fe1eaba4d88aa400c34b4d5791b/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:59.612476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252edbc-801e-4904-abf7-cd2a7b6624b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9dd8fde445f5496fab61c82990f8ba09, user_id=52894fe1eaba4d88aa400c34b4d5791b, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:50:59.614376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252edbc-801e-4904-abf7-cd2a7b6624b6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:50:59.649385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 58/125] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:50:59 2022] PUT /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09/users/52894fe1eaba4d88aa400c34b4d5791b/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:50:59.658959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252edbc-801e-4904-abf7-cd2a7b6624b6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:59.672626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef6dd1-fadb-4906-be23-570115d99391 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:59.760293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a114e2-6c19-49d7-a864-e89bd2ad9a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:59.761091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a114e2-6c19-49d7-a864-e89bd2ad9a81 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.761381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a114e2-6c19-49d7-a864-e89bd2ad9a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.763792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a114e2-6c19-49d7-a864-e89bd2ad9a81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:50:59.794991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a114e2-6c19-49d7-a864-e89bd2ad9a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d9a114e2-6c19-49d7-a864-e89bd2ad9a81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:50:59.820153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:50:59.821096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:50:59.821836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 68/126] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:50:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:50:59.831036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a114e2-6c19-49d7-a864-e89bd2ad9a81 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:50:59.933000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252edbc-801e-4904-abf7-cd2a7b6624b6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:50:59.993178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d3cc2-b794-41be-8d33-f9c101f06017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:50:59.994103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d3cc2-b794-41be-8d33-f9c101f06017 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:50:59.994389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d3cc2-b794-41be-8d33-f9c101f06017 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:50:59.994675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d3cc2-b794-41be-8d33-f9c101f06017 None admin] PATH_INFO: `/v3/users/6fdb844dddc042b9958d0532be8d63c7` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:00.002716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d3cc2-b794-41be-8d33-f9c101f06017 None admin] RBAC: Authorizing `identity:get_user(user_id=6fdb844dddc042b9958d0532be8d63c7)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:00.004221 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d3cc2-b794-41be-8d33-f9c101f06017 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:00.006208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 59/127] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:50:59 2022] GET /identity/v3/users/6fdb844dddc042b9958d0532be8d63c7 => generated 275 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:00.024383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d3cc2-b794-41be-8d33-f9c101f06017 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:00.121016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a114e2-6c19-49d7-a864-e89bd2ad9a81 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:00.124844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:00.215611 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3112c-6a30-4ffc-9131-8a810732edb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:00.222109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3112c-6a30-4ffc-9131-8a810732edb1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:00.222109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3112c-6a30-4ffc-9131-8a810732edb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:00.222109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3112c-6a30-4ffc-9131-8a810732edb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:00.222109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3112c-6a30-4ffc-9131-8a810732edb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:00.223236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3112c-6a30-4ffc-9131-8a810732edb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:00.224750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3112c-6a30-4ffc-9131-8a810732edb1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:00.238996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:00.246020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 69/128] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:50:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:00.246956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3112c-6a30-4ffc-9131-8a810732edb1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:00.258059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d3cc2-b794-41be-8d33-f9c101f06017 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:00.301743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f54c8-4e33-4a20-a033-76bbdf956e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:00.302519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f54c8-4e33-4a20-a033-76bbdf956e65 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:00.302807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f54c8-4e33-4a20-a033-76bbdf956e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:00.303086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f54c8-4e33-4a20-a033-76bbdf956e65 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:00.303853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f54c8-4e33-4a20-a033-76bbdf956e65 None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:00.304167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f54c8-4e33-4a20-a033-76bbdf956e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:00.305760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f54c8-4e33-4a20-a033-76bbdf956e65 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:00.328342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 60/129] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:00.329654 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f54c8-4e33-4a20-a033-76bbdf956e65 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:00.391576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3112c-6a30-4ffc-9131-8a810732edb1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:00.462239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe68383-8e85-48ba-aeff-b2567c612410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:00.463188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe68383-8e85-48ba-aeff-b2567c612410 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:00.463483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe68383-8e85-48ba-aeff-b2567c612410 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:00.463854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe68383-8e85-48ba-aeff-b2567c612410 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:00.471021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe68383-8e85-48ba-aeff-b2567c612410 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:00.472615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe68383-8e85-48ba-aeff-b2567c612410 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:00.474542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 70/130] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:00 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:00.475333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe68383-8e85-48ba-aeff-b2567c612410 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:00.480456 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:00.497025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f54c8-4e33-4a20-a033-76bbdf956e65 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:00.572137 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c05bc5-b6a1-483e-80d7-527b56eae0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:00.573012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c05bc5-b6a1-483e-80d7-527b56eae0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:00.573305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c05bc5-b6a1-483e-80d7-527b56eae0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:00.573637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c05bc5-b6a1-483e-80d7-527b56eae0b5 None admin] PATH_INFO: `/v3/users/6fdb844dddc042b9958d0532be8d63c7` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:00.575143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c05bc5-b6a1-483e-80d7-527b56eae0b5 None admin] RBAC: Authorizing `identity:get_user(user_id=6fdb844dddc042b9958d0532be8d63c7)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:00.576643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c05bc5-b6a1-483e-80d7-527b56eae0b5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:00.578568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 61/131] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:00 2022] GET /identity/v3/users/6fdb844dddc042b9958d0532be8d63c7 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:00.579091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c05bc5-b6a1-483e-80d7-527b56eae0b5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:00.583465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:00.628976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe68383-8e85-48ba-aeff-b2567c612410 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:00.699767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8e2b-c259-425d-8839-0e3c5434d6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:00.700668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8e2b-c259-425d-8839-0e3c5434d6ab None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:00.700950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8e2b-c259-425d-8839-0e3c5434d6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:00.701224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8e2b-c259-425d-8839-0e3c5434d6ab None admin] PATH_INFO: `/v3/users/6fdb844dddc042b9958d0532be8d63c7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:00.702636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f8e2b-c259-425d-8839-0e3c5434d6ab None admin] RBAC: Authorizing `identity:get_user(user_id=6fdb844dddc042b9958d0532be8d63c7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:00.704986 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f8e2b-c259-425d-8839-0e3c5434d6ab None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:00.707571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 71/132] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:00 2022] GET /identity/v3/users/6fdb844dddc042b9958d0532be8d63c7 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:00.710059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8e2b-c259-425d-8839-0e3c5434d6ab None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:00.790220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c05bc5-b6a1-483e-80d7-527b56eae0b5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:00.867471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8e2b-c259-425d-8839-0e3c5434d6ab None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:00.874773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3babf9-9e63-4110-8496-24939dacfe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:00.875760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3babf9-9e63-4110-8496-24939dacfe23 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:00.876051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3babf9-9e63-4110-8496-24939dacfe23 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:00.876335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3babf9-9e63-4110-8496-24939dacfe23 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09/users/6fdb844dddc042b9958d0532be8d63c7/roles/ffab93457c0f4f64a29764f17759396e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:00.886907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3babf9-9e63-4110-8496-24939dacfe23 None admin] RBAC: Authorizing `identity:create_grant(project_id=9dd8fde445f5496fab61c82990f8ba09, user_id=6fdb844dddc042b9958d0532be8d63c7, role_id=ffab93457c0f4f64a29764f17759396e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:00.888510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3babf9-9e63-4110-8496-24939dacfe23 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:00.905214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50f8ab-3e7d-4e5c-a666-9b04d6ac1dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:00.905214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50f8ab-3e7d-4e5c-a666-9b04d6ac1dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:00.905214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50f8ab-3e7d-4e5c-a666-9b04d6ac1dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:00.905544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50f8ab-3e7d-4e5c-a666-9b04d6ac1dc2 None admin] PATH_INFO: `/v3/projects/73b2d8f446f84ec69cdd45be83d60f38` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:00.906774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50f8ab-3e7d-4e5c-a666-9b04d6ac1dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=73b2d8f446f84ec69cdd45be83d60f38)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:00.908296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50f8ab-3e7d-4e5c-a666-9b04d6ac1dc2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:00.912585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 72/133] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:00 2022] GET /identity/v3/projects/73b2d8f446f84ec69cdd45be83d60f38 => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:00.912585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50f8ab-3e7d-4e5c-a666-9b04d6ac1dc2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:00.922506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:00.924290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 62/134] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:00 2022] PUT /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09/users/6fdb844dddc042b9958d0532be8d63c7/roles/ffab93457c0f4f64a29764f17759396e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:00.925182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3babf9-9e63-4110-8496-24939dacfe23 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:00.979889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:01.154524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3babf9-9e63-4110-8496-24939dacfe23 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:01.157156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50f8ab-3e7d-4e5c-a666-9b04d6ac1dc2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:01.192178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ca925-ee8c-416c-b303-b287068248b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:01.193100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ca925-ee8c-416c-b303-b287068248b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:01.193409 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ca925-ee8c-416c-b303-b287068248b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:01.193677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ca925-ee8c-416c-b303-b287068248b0 None admin] PATH_INFO: `/v3/projects/73b2d8f446f84ec69cdd45be83d60f38/users/6fdb844dddc042b9958d0532be8d63c7/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:01.195751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ca925-ee8c-416c-b303-b287068248b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=73b2d8f446f84ec69cdd45be83d60f38, user_id=6fdb844dddc042b9958d0532be8d63c7, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:01.197255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ca925-ee8c-416c-b303-b287068248b0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:01.206216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b77489-5ffc-4f83-9bf9-701e2b182242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:01.207474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77489-5ffc-4f83-9bf9-701e2b182242 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:01.207767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77489-5ffc-4f83-9bf9-701e2b182242 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:01.208110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77489-5ffc-4f83-9bf9-701e2b182242 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:01.209712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b77489-5ffc-4f83-9bf9-701e2b182242 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:01.211607 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b77489-5ffc-4f83-9bf9-701e2b182242 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:01.213926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 73/135] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:00 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:01.215291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608a47a-337e-462e-a7e7-e12eaa509ff2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:01.215735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608a47a-337e-462e-a7e7-e12eaa509ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:01.216086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 63/136] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:00 2022] PUT /identity/v3/projects/73b2d8f446f84ec69cdd45be83d60f38/users/6fdb844dddc042b9958d0532be8d63c7/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:01.220713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608a47a-337e-462e-a7e7-e12eaa509ff2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:01.221793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ca925-ee8c-416c-b303-b287068248b0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:01.222387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 74/137] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:01.230066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ab1b4-0fc7-432c-9737-eddb7225c643 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:01.231079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ab1b4-0fc7-432c-9737-eddb7225c643 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:01.232880 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ab1b4-0fc7-432c-9737-eddb7225c643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:01.260914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3a6ab1b4-0fc7-432c-9737-eddb7225c643 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:01.262610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3a6ab1b4-0fc7-432c-9737-eddb7225c643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:01.368148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 75/138] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:01.369340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:01.374491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5381-556f-4c79-8369-a380b90c2f48 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:01.374889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5381-556f-4c79-8369-a380b90c2f48 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:01.375184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5381-556f-4c79-8369-a380b90c2f48 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:01.376895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 76/139] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:01.387772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91bcc5-3c59-4f4f-a031-751f0cadcac7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:01.388145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91bcc5-3c59-4f4f-a031-751f0cadcac7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:01.388348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91bcc5-3c59-4f4f-a031-751f0cadcac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:01.394184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:01.398548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ca925-ee8c-416c-b303-b287068248b0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:01.417674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6e91bcc5-3c59-4f4f-a031-751f0cadcac7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:01.419029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6e91bcc5-3c59-4f4f-a031-751f0cadcac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:01.439830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6e163-10db-4297-a7a0-27190687f6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:01.441849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6e163-10db-4297-a7a0-27190687f6fc None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:01.442840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6e163-10db-4297-a7a0-27190687f6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:01.443138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6e163-10db-4297-a7a0-27190687f6fc None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09/users/6fdb844dddc042b9958d0532be8d63c7/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:01.445252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6e163-10db-4297-a7a0-27190687f6fc None admin] RBAC: Authorizing `identity:create_grant(project_id=9dd8fde445f5496fab61c82990f8ba09, user_id=6fdb844dddc042b9958d0532be8d63c7, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:01.447301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6e163-10db-4297-a7a0-27190687f6fc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:01.467087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 64/140] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:01 2022] PUT /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09/users/6fdb844dddc042b9958d0532be8d63c7/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:01.468035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6e163-10db-4297-a7a0-27190687f6fc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:01.516497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 77/141] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:01.562102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91bcc5-3c59-4f4f-a031-751f0cadcac7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:01.590604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6e163-10db-4297-a7a0-27190687f6fc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:01.660417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263ce6a-311a-4301-bb4b-197f28d19fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:01.662393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263ce6a-311a-4301-bb4b-197f28d19fbf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:01.663188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263ce6a-311a-4301-bb4b-197f28d19fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:01.664598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263ce6a-311a-4301-bb4b-197f28d19fbf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:01.687999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263ce6a-311a-4301-bb4b-197f28d19fbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5263ce6a-311a-4301-bb4b-197f28d19fbf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:01.703643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:01.704555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:01.705299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:01.709545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 65/142] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:01.717196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263ce6a-311a-4301-bb4b-197f28d19fbf None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:01.725809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:01.768707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91bcc5-3c59-4f4f-a031-751f0cadcac7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:01.803555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:01.811271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3b7f-d59d-44f8-8407-0dbb40bdd6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:01.812083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3b7f-d59d-44f8-8407-0dbb40bdd6af None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:01.812377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3b7f-d59d-44f8-8407-0dbb40bdd6af None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:01.812711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e3b7f-d59d-44f8-8407-0dbb40bdd6af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:01.813931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e3b7f-d59d-44f8-8407-0dbb40bdd6af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:01.815778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e3b7f-d59d-44f8-8407-0dbb40bdd6af None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:01.819198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 78/143] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:01.825164 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3b7f-d59d-44f8-8407-0dbb40bdd6af None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:01.891437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263ce6a-311a-4301-bb4b-197f28d19fbf None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:01.928088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939bd1c-8f78-4c3c-9cf3-22116cf6096a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:01.928770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939bd1c-8f78-4c3c-9cf3-22116cf6096a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:01.929056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939bd1c-8f78-4c3c-9cf3-22116cf6096a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:01.929638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c939bd1c-8f78-4c3c-9cf3-22116cf6096a None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:01.930227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939bd1c-8f78-4c3c-9cf3-22116cf6096a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:01.930542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939bd1c-8f78-4c3c-9cf3-22116cf6096a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:01.932009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c939bd1c-8f78-4c3c-9cf3-22116cf6096a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:01.942860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 66/144] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:01.948238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939bd1c-8f78-4c3c-9cf3-22116cf6096a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:01.955495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e3b7f-d59d-44f8-8407-0dbb40bdd6af None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:02.001980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4f385-ef94-477b-a3fd-86f071958c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:02.003086 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4f385-ef94-477b-a3fd-86f071958c1c None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:02.003575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4f385-ef94-477b-a3fd-86f071958c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:02.004041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4f385-ef94-477b-a3fd-86f071958c1c None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:02.004963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4f385-ef94-477b-a3fd-86f071958c1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:02.006666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4f385-ef94-477b-a3fd-86f071958c1c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:02.020162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:02.053359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 79/145] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:51:01 2022] POST /identity/v3/users => generated 283 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 10:51:02.071914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c939bd1c-8f78-4c3c-9cf3-22116cf6096a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:02.114448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10b6eb-443b-4021-bacd-7633639e5b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:02.115650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10b6eb-443b-4021-bacd-7633639e5b63 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:02.116072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10b6eb-443b-4021-bacd-7633639e5b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:02.116506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10b6eb-443b-4021-bacd-7633639e5b63 None admin] PATH_INFO: `/v3/users/c4793dfbf0514b038e9957c59546b023` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:02.118012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10b6eb-443b-4021-bacd-7633639e5b63 None admin] RBAC: Authorizing `identity:get_user(user_id=c4793dfbf0514b038e9957c59546b023)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:02.119724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10b6eb-443b-4021-bacd-7633639e5b63 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:02.121765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 67/146] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:01 2022] GET /identity/v3/users/c4793dfbf0514b038e9957c59546b023 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:02.127806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4f385-ef94-477b-a3fd-86f071958c1c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:02.134904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57d333-32b9-4bd0-ae79-b6a0941fc25d None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:02.135510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57d333-32b9-4bd0-ae79-b6a0941fc25d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:02.135922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57d333-32b9-4bd0-ae79-b6a0941fc25d None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:02.137576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 68/147] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:51:02.143248 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fa7e7-02f5-47e0-a469-6a9389a8e7a9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:02.143788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fa7e7-02f5-47e0-a469-6a9389a8e7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:02.144198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727fa7e7-02f5-47e0-a469-6a9389a8e7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:02.175265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-727fa7e7-02f5-47e0-a469-6a9389a8e7a9 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:02.177533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-727fa7e7-02f5-47e0-a469-6a9389a8e7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:02.267429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:02.281551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 69/148] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:02.316879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fa7e7-02f5-47e0-a469-6a9389a8e7a9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:02.382834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4f385-ef94-477b-a3fd-86f071958c1c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:02.409480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:02.456456 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36f12e-52ac-4ff5-b47f-2113c1e65a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:02.457343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36f12e-52ac-4ff5-b47f-2113c1e65a0b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:02.457620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36f12e-52ac-4ff5-b47f-2113c1e65a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:02.457896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36f12e-52ac-4ff5-b47f-2113c1e65a0b None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:02.462258 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36f12e-52ac-4ff5-b47f-2113c1e65a0b None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:02.463756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36f12e-52ac-4ff5-b47f-2113c1e65a0b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:02.465563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 80/149] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:02 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:02.472033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36f12e-52ac-4ff5-b47f-2113c1e65a0b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:02.491302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727fa7e7-02f5-47e0-a469-6a9389a8e7a9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:02.558610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d7021-6d78-417c-ad3e-8c85b83292ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:02.559473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d7021-6d78-417c-ad3e-8c85b83292ef None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:02.559759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d7021-6d78-417c-ad3e-8c85b83292ef None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:02.560037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d7021-6d78-417c-ad3e-8c85b83292ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:02.597456 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d7021-6d78-417c-ad3e-8c85b83292ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2f8d7021-6d78-417c-ad3e-8c85b83292ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:02.608608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:02.609500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:02.610343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 70/150] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:51:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:02.617994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d7021-6d78-417c-ad3e-8c85b83292ef None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:02.672611 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36f12e-52ac-4ff5-b47f-2113c1e65a0b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:02.710381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:02.732281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7dbe4-194c-4b67-b0aa-0eb04c2e6940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:02.733596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7dbe4-194c-4b67-b0aa-0eb04c2e6940 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:02.734130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7dbe4-194c-4b67-b0aa-0eb04c2e6940 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:02.734619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7dbe4-194c-4b67-b0aa-0eb04c2e6940 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09/users/c4793dfbf0514b038e9957c59546b023/roles/e4214ef8456541528194f90e229b356a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:02.741731 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7dbe4-194c-4b67-b0aa-0eb04c2e6940 None admin] RBAC: Authorizing `identity:create_grant(project_id=9dd8fde445f5496fab61c82990f8ba09, user_id=c4793dfbf0514b038e9957c59546b023, role_id=e4214ef8456541528194f90e229b356a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:02.743512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7dbe4-194c-4b67-b0aa-0eb04c2e6940 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:02.766020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 81/151] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:02 2022] PUT /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09/users/c4793dfbf0514b038e9957c59546b023/roles/e4214ef8456541528194f90e229b356a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:02.782068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d7021-6d78-417c-ad3e-8c85b83292ef None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:02.807413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:02.822167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc36cba-b15c-45d9-a649-25bf4f933e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:02.822830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc36cba-b15c-45d9-a649-25bf4f933e53 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:02.823166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc36cba-b15c-45d9-a649-25bf4f933e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:02.823452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc36cba-b15c-45d9-a649-25bf4f933e53 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:02.824227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc36cba-b15c-45d9-a649-25bf4f933e53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:02.824640 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc36cba-b15c-45d9-a649-25bf4f933e53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:02.826296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc36cba-b15c-45d9-a649-25bf4f933e53 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:02.839114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 71/152] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:02.842346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7dbe4-194c-4b67-b0aa-0eb04c2e6940 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:02.972316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7dbe4-194c-4b67-b0aa-0eb04c2e6940 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:03.011384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea35a4c-0301-4e3d-9ea0-4107fd46b386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:03.012549 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea35a4c-0301-4e3d-9ea0-4107fd46b386 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:03.012924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea35a4c-0301-4e3d-9ea0-4107fd46b386 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:03.013307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea35a4c-0301-4e3d-9ea0-4107fd46b386 None admin] PATH_INFO: `/v3/users/52894fe1eaba4d88aa400c34b4d5791b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:03.018948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea35a4c-0301-4e3d-9ea0-4107fd46b386 None admin] RBAC: Authorizing `identity:get_user(user_id=52894fe1eaba4d88aa400c34b4d5791b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:03.020456 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea35a4c-0301-4e3d-9ea0-4107fd46b386 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:03.022294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 82/153] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:02 2022] GET /identity/v3/users/52894fe1eaba4d88aa400c34b4d5791b => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:03.028202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc36cba-b15c-45d9-a649-25bf4f933e53 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:03.150943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc36cba-b15c-45d9-a649-25bf4f933e53 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:03.192256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb973e-3a4c-43be-a9c3-fa2a5d0dba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:03.192926 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb973e-3a4c-43be-a9c3-fa2a5d0dba49 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:03.193209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb973e-3a4c-43be-a9c3-fa2a5d0dba49 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:03.193487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb973e-3a4c-43be-a9c3-fa2a5d0dba49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:03.261036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:03.279327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb973e-3a4c-43be-a9c3-fa2a5d0dba49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:03.280833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb973e-3a4c-43be-a9c3-fa2a5d0dba49 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:03.388803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 72/154] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:03.394678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea35a4c-0301-4e3d-9ea0-4107fd46b386 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:03.451154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:03.508943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea35a4c-0301-4e3d-9ea0-4107fd46b386 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:03.545849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cee73c-3b80-4d0b-b7c1-62e3f9182d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:03.546770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cee73c-3b80-4d0b-b7c1-62e3f9182d25 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:03.547037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cee73c-3b80-4d0b-b7c1-62e3f9182d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:03.547308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cee73c-3b80-4d0b-b7c1-62e3f9182d25 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:03.548567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cee73c-3b80-4d0b-b7c1-62e3f9182d25 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:03.550031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cee73c-3b80-4d0b-b7c1-62e3f9182d25 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:03.553069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 83/155] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:03 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:03.558071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb973e-3a4c-43be-a9c3-fa2a5d0dba49 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:03.662779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:03.682482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb973e-3a4c-43be-a9c3-fa2a5d0dba49 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:03.724948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea7b0e-0bf0-4b49-95cc-4d36f65e3921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:03.725534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea7b0e-0bf0-4b49-95cc-4d36f65e3921 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:03.725814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea7b0e-0bf0-4b49-95cc-4d36f65e3921 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:03.726105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea7b0e-0bf0-4b49-95cc-4d36f65e3921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:03.726991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea7b0e-0bf0-4b49-95cc-4d36f65e3921 None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=9dd8fde445f5496fab61c82990f8ba09, user.id=52894fe1eaba4d88aa400c34b4d5791b) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:03.727306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea7b0e-0bf0-4b49-95cc-4d36f65e3921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:03.728790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea7b0e-0bf0-4b49-95cc-4d36f65e3921 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:03.737788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 73/156] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:03 2022] GET /identity/v3/role_assignments?user.id=52894fe1eaba4d88aa400c34b4d5791b&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:04.673547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e42e5b-1b5e-4578-a5f9-da3e46c3d357 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:04.674059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e42e5b-1b5e-4578-a5f9-da3e46c3d357 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:04.674155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e42e5b-1b5e-4578-a5f9-da3e46c3d357 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:04.675748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 84/157] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:04.681170 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00694458-3862-458e-ad2c-b4f48234c590 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:04.681660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00694458-3862-458e-ad2c-b4f48234c590 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:04.681854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00694458-3862-458e-ad2c-b4f48234c590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:04.683629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18c1e1-57ce-44f4-abb6-5c3fdfb50705 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:04.683958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18c1e1-57ce-44f4-abb6-5c3fdfb50705 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:04.684281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18c1e1-57ce-44f4-abb6-5c3fdfb50705 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:04.685858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 85/158] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:51:04.693685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ac839-4610-4040-997b-357ca510c864 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:04.693963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ac839-4610-4040-997b-357ca510c864 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:04.694277 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ac839-4610-4040-997b-357ca510c864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:04.726400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-834ac839-4610-4040-997b-357ca510c864 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:04.727668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-834ac839-4610-4040-997b-357ca510c864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:04.734681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-00694458-3862-458e-ad2c-b4f48234c590 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:04.735898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-00694458-3862-458e-ad2c-b4f48234c590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:04.867158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 86/159] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:04.871076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 74/160] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:04.895039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ac839-4610-4040-997b-357ca510c864 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:04.908552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00694458-3862-458e-ad2c-b4f48234c590 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:05.045487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:05.060181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:05.127775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ac839-4610-4040-997b-357ca510c864 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:05.130013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00694458-3862-458e-ad2c-b4f48234c590 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:05.183622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752cabd-2b60-4fa2-bb23-b83785f80880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:05.184989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752cabd-2b60-4fa2-bb23-b83785f80880 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:05.185269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752cabd-2b60-4fa2-bb23-b83785f80880 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:05.185541 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752cabd-2b60-4fa2-bb23-b83785f80880 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:05.196587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45c429-9838-468d-8980-5a4597f3614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:05.197704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45c429-9838-468d-8980-5a4597f3614c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:05.198211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45c429-9838-468d-8980-5a4597f3614c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:05.198813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45c429-9838-468d-8980-5a4597f3614c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:05.203789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752cabd-2b60-4fa2-bb23-b83785f80880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:05.210352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45c429-9838-468d-8980-5a4597f3614c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0e45c429-9838-468d-8980-5a4597f3614c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:05.218482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:05.219319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:05.220074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:05.222506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 75/161] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e752cabd-2b60-4fa2-bb23-b83785f80880 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:05.224824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:05.225650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:05.226502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 87/162] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:05.232231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45c429-9838-468d-8980-5a4597f3614c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:05.243754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752cabd-2b60-4fa2-bb23-b83785f80880 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:05.357355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752cabd-2b60-4fa2-bb23-b83785f80880 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:05.374142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:05.379841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45c429-9838-468d-8980-5a4597f3614c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:05.410923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49556a7a-0751-468d-9b62-7c22f446dfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:05.411603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49556a7a-0751-468d-9b62-7c22f446dfea None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:05.411881 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49556a7a-0751-468d-9b62-7c22f446dfea None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:05.412152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49556a7a-0751-468d-9b62-7c22f446dfea None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:05.412896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49556a7a-0751-468d-9b62-7c22f446dfea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:05.413201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49556a7a-0751-468d-9b62-7c22f446dfea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:05.414640 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49556a7a-0751-468d-9b62-7c22f446dfea None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:05.431400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 88/163] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:05.437947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49556a7a-0751-468d-9b62-7c22f446dfea None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:05.454189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1702ae-9592-4356-9d3d-66884fdb27ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:05.454876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1702ae-9592-4356-9d3d-66884fdb27ba None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:05.455159 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1702ae-9592-4356-9d3d-66884fdb27ba None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:05.455432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1702ae-9592-4356-9d3d-66884fdb27ba None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:05.456197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1702ae-9592-4356-9d3d-66884fdb27ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:05.456502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1702ae-9592-4356-9d3d-66884fdb27ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:05.457900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1702ae-9592-4356-9d3d-66884fdb27ba None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:05.470701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:05.480761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 76/164] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:05.485853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1702ae-9592-4356-9d3d-66884fdb27ba None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:05.574834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49556a7a-0751-468d-9b62-7c22f446dfea None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:05.643860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038015af-bf71-4752-9921-6c8eca5e2d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:05.644676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038015af-bf71-4752-9921-6c8eca5e2d66 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:05.644958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038015af-bf71-4752-9921-6c8eca5e2d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:05.645236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038015af-bf71-4752-9921-6c8eca5e2d66 None admin] PATH_INFO: `/v3/users/6fdb844dddc042b9958d0532be8d63c7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:05.650823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038015af-bf71-4752-9921-6c8eca5e2d66 None admin] RBAC: Authorizing `identity:get_user(user_id=6fdb844dddc042b9958d0532be8d63c7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:05.652302 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038015af-bf71-4752-9921-6c8eca5e2d66 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:05.654639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 89/165] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:05 2022] GET /identity/v3/users/6fdb844dddc042b9958d0532be8d63c7 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:05.655929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063bae6-2ad6-49e1-aa69-56694d2e8fd9 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:05.656215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063bae6-2ad6-49e1-aa69-56694d2e8fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:05.656490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063bae6-2ad6-49e1-aa69-56694d2e8fd9 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:05.657905 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 90/166] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:05.661188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063bae6-2ad6-49e1-aa69-56694d2e8fd9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:05.708083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1702ae-9592-4356-9d3d-66884fdb27ba None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:05.730438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:05.799715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105c441-8abc-4101-8ca1-2072770ea99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:05.800618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105c441-8abc-4101-8ca1-2072770ea99b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:05.800903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105c441-8abc-4101-8ca1-2072770ea99b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:05.801182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105c441-8abc-4101-8ca1-2072770ea99b None admin] PATH_INFO: `/v3/users/6fdb844dddc042b9958d0532be8d63c7` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:05.802474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105c441-8abc-4101-8ca1-2072770ea99b None admin] RBAC: Authorizing `identity:get_user(user_id=6fdb844dddc042b9958d0532be8d63c7)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:05.803994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105c441-8abc-4101-8ca1-2072770ea99b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:05.805667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 77/167] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:05 2022] GET /identity/v3/users/6fdb844dddc042b9958d0532be8d63c7 => generated 275 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:05.811429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f4d68-a559-4c39-b320-f5665ba746ff None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:05.811719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f4d68-a559-4c39-b320-f5665ba746ff None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:05.812003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f4d68-a559-4c39-b320-f5665ba746ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:05.843852 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-180f4d68-a559-4c39-b320-f5665ba746ff None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:05.845046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-180f4d68-a559-4c39-b320-f5665ba746ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:05.867359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063bae6-2ad6-49e1-aa69-56694d2e8fd9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:05.939401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f27107-f748-41c8-a3df-58ef0245970d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:05.940128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f27107-f748-41c8-a3df-58ef0245970d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:05.940408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f27107-f748-41c8-a3df-58ef0245970d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:05.940684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f27107-f748-41c8-a3df-58ef0245970d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.005854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:06.017294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 78/168] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:06.018185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f4d68-a559-4c39-b320-f5665ba746ff None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:06.098829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f27107-f748-41c8-a3df-58ef0245970d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:06.100326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f27107-f748-41c8-a3df-58ef0245970d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:06.150927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:06.234780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f4d68-a559-4c39-b320-f5665ba746ff None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:06.245128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 91/169] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 10:51:06.246005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f27107-f748-41c8-a3df-58ef0245970d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:06.275898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfc0a2-2943-4884-9c0b-5d4ea9a16794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:06.276659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc0a2-2943-4884-9c0b-5d4ea9a16794 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:06.276947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc0a2-2943-4884-9c0b-5d4ea9a16794 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:06.277228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfc0a2-2943-4884-9c0b-5d4ea9a16794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.409408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:06.415129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc0a2-2943-4884-9c0b-5d4ea9a16794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:06.416758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfc0a2-2943-4884-9c0b-5d4ea9a16794 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:06.422498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f27107-f748-41c8-a3df-58ef0245970d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:06.514928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba66580-3a56-4df4-965d-282eb30b38c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:06.515678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba66580-3a56-4df4-965d-282eb30b38c5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:06.515960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba66580-3a56-4df4-965d-282eb30b38c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:06.516235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba66580-3a56-4df4-965d-282eb30b38c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.536639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:06.546727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba66580-3a56-4df4-965d-282eb30b38c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-bba66580-3a56-4df4-965d-282eb30b38c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:06.554534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:06.555417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:06.556185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 92/170] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:06.557292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba66580-3a56-4df4-965d-282eb30b38c5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:06.582215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 79/171] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:06.584147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9318b-d889-441a-953e-8f290a81d402 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:06.584439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9318b-d889-441a-953e-8f290a81d402 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:06.584764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9318b-d889-441a-953e-8f290a81d402 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.586059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 80/172] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:06.586825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9318b-d889-441a-953e-8f290a81d402 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:06.745929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba66580-3a56-4df4-965d-282eb30b38c5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:06.757459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:06.763454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9318b-d889-441a-953e-8f290a81d402 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:06.780208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee7f0e-5705-418c-9400-fb689ccc51ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:06.781088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7f0e-5705-418c-9400-fb689ccc51ac None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:06.781373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7f0e-5705-418c-9400-fb689ccc51ac None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:06.781650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7f0e-5705-418c-9400-fb689ccc51ac None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.782858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee7f0e-5705-418c-9400-fb689ccc51ac None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:06.784326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee7f0e-5705-418c-9400-fb689ccc51ac None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:06.786032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 93/173] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:06 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:06.786830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee7f0e-5705-418c-9400-fb689ccc51ac None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:06.803119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa296af0-012e-4dfd-a463-b5eed3d6febf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:06.803803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa296af0-012e-4dfd-a463-b5eed3d6febf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:06.804087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa296af0-012e-4dfd-a463-b5eed3d6febf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:06.804370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa296af0-012e-4dfd-a463-b5eed3d6febf None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.805129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa296af0-012e-4dfd-a463-b5eed3d6febf None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:06.805439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa296af0-012e-4dfd-a463-b5eed3d6febf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:06.806871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa296af0-012e-4dfd-a463-b5eed3d6febf None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:06.815110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 81/174] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:06.819353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ff1f3-4cdb-49b8-b806-3ba85dcba116 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:06.819824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ff1f3-4cdb-49b8-b806-3ba85dcba116 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:06.820134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ff1f3-4cdb-49b8-b806-3ba85dcba116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.836066 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8f8ff1f3-4cdb-49b8-b806-3ba85dcba116 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:06.837241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8f8ff1f3-4cdb-49b8-b806-3ba85dcba116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:06.854053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:06.887189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee7f0e-5705-418c-9400-fb689ccc51ac None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:06.903132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 82/175] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:06.904485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4879f-c79c-46c8-9871-d63c7ef9e7e5 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:06.904788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4879f-c79c-46c8-9871-d63c7ef9e7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:06.906453 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4879f-c79c-46c8-9871-d63c7ef9e7e5 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.911698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 83/176] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:06.911698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36d048-e6da-476f-a5a1-401eb94a4fc8 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:06.911698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36d048-e6da-476f-a5a1-401eb94a4fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:06.911698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36d048-e6da-476f-a5a1-401eb94a4fc8 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.911698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 84/177] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:06.914323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcd13d-e984-4ba1-82cf-d6387a342c54 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:06.914922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcd13d-e984-4ba1-82cf-d6387a342c54 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:06.915383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcd13d-e984-4ba1-82cf-d6387a342c54 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.917799 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 85/178] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:06.919501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcd13d-e984-4ba1-82cf-d6387a342c54 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:06.929113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eff19e-edd3-440a-9a7b-59a177b4e1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:06.930101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eff19e-edd3-440a-9a7b-59a177b4e1de None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:06.930398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eff19e-edd3-440a-9a7b-59a177b4e1de None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:06.930672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eff19e-edd3-440a-9a7b-59a177b4e1de None admin] PATH_INFO: `/v3/projects/73b2d8f446f84ec69cdd45be83d60f38` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:06.931868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eff19e-edd3-440a-9a7b-59a177b4e1de None admin] RBAC: Authorizing `identity:get_project(project_id=73b2d8f446f84ec69cdd45be83d60f38)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:06.933340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eff19e-edd3-440a-9a7b-59a177b4e1de None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:06.935187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 94/179] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:06 2022] GET /identity/v3/projects/73b2d8f446f84ec69cdd45be83d60f38 => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:06.935941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eff19e-edd3-440a-9a7b-59a177b4e1de None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:07.016481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:07.030094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcd13d-e984-4ba1-82cf-d6387a342c54 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:07.035638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eff19e-edd3-440a-9a7b-59a177b4e1de None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:07.063552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84485938-6e3a-4a0d-87cb-d143ecd18d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:07.064205 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84485938-6e3a-4a0d-87cb-d143ecd18d2c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.064484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84485938-6e3a-4a0d-87cb-d143ecd18d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.064754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84485938-6e3a-4a0d-87cb-d143ecd18d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.065594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84485938-6e3a-4a0d-87cb-d143ecd18d2c None admin] RBAC: Adding query filter params (role.id=ffab93457c0f4f64a29764f17759396e, scope.project.id=9dd8fde445f5496fab61c82990f8ba09, user.id=6fdb844dddc042b9958d0532be8d63c7) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:07.065904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84485938-6e3a-4a0d-87cb-d143ecd18d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:07.067469 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84485938-6e3a-4a0d-87cb-d143ecd18d2c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:07.069198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e16e68-13cd-4ce6-80c7-d09991a2c8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:07.069974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e16e68-13cd-4ce6-80c7-d09991a2c8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.070288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e16e68-13cd-4ce6-80c7-d09991a2c8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.070633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e16e68-13cd-4ce6-80c7-d09991a2c8d5 None admin] PATH_INFO: `/v3/users/6fdb844dddc042b9958d0532be8d63c7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.071831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e16e68-13cd-4ce6-80c7-d09991a2c8d5 None admin] RBAC: Authorizing `identity:get_user(user_id=6fdb844dddc042b9958d0532be8d63c7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:07.073243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 86/180] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:06 2022] GET /identity/v3/role_assignments?user.id=6fdb844dddc042b9958d0532be8d63c7&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=ffab93457c0f4f64a29764f17759396e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:07.073360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e16e68-13cd-4ce6-80c7-d09991a2c8d5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:07.074473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727374fc-d55f-4178-bca2-a74efa85d3de None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.074845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727374fc-d55f-4178-bca2-a74efa85d3de None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.075293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727374fc-d55f-4178-bca2-a74efa85d3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.075382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 95/181] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:06 2022] GET /identity/v3/users/6fdb844dddc042b9958d0532be8d63c7 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:07.079292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc247b9-7c0b-442a-b67e-b4f57eff4bae None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.080158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc247b9-7c0b-442a-b67e-b4f57eff4bae None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.080656 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc247b9-7c0b-442a-b67e-b4f57eff4bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.092410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:07.092874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-727374fc-d55f-4178-bca2-a74efa85d3de None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:07.094138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-727374fc-d55f-4178-bca2-a74efa85d3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:07.105404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1bc247b9-7c0b-442a-b67e-b4f57eff4bae None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:07.106775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1bc247b9-7c0b-442a-b67e-b4f57eff4bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:07.170497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 87/182] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:07.176148 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0ffd6-44a8-4509-9474-8cba0b30c475 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.176429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0ffd6-44a8-4509-9474-8cba0b30c475 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.176749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0ffd6-44a8-4509-9474-8cba0b30c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.190966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 96/183] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:07.202278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc247b9-7c0b-442a-b67e-b4f57eff4bae None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:07.202364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f7c0ffd6-44a8-4509-9474-8cba0b30c475 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:07.202364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f7c0ffd6-44a8-4509-9474-8cba0b30c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:07.285403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 88/184] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:07.286398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0ffd6-44a8-4509-9474-8cba0b30c475 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:07.303141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:07.314108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc247b9-7c0b-442a-b67e-b4f57eff4bae None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:07.349218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c029ec-2172-4af6-b2ac-80bb7c584581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:07.350036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c029ec-2172-4af6-b2ac-80bb7c584581 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.350345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c029ec-2172-4af6-b2ac-80bb7c584581 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.350614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c029ec-2172-4af6-b2ac-80bb7c584581 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.352454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:07.361993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c029ec-2172-4af6-b2ac-80bb7c584581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-00c029ec-2172-4af6-b2ac-80bb7c584581 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:07.370001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:07.371094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:07.372083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 97/185] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:07.372778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c029ec-2172-4af6-b2ac-80bb7c584581 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:07.394763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0ffd6-44a8-4509-9474-8cba0b30c475 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:07.431705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb8c04-f5db-48d0-a719-1274fa104f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:07.432372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb8c04-f5db-48d0-a719-1274fa104f90 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.432652 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb8c04-f5db-48d0-a719-1274fa104f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.432924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb8c04-f5db-48d0-a719-1274fa104f90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.433828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb8c04-f5db-48d0-a719-1274fa104f90 None admin] RBAC: Adding query filter params (role.id=3e46d43e30ef494087da2c44f7ca702f, scope.project.id=73b2d8f446f84ec69cdd45be83d60f38, user.id=6fdb844dddc042b9958d0532be8d63c7) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:07.434173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb8c04-f5db-48d0-a719-1274fa104f90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:07.435662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb8c04-f5db-48d0-a719-1274fa104f90 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:07.441528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 89/186] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:07 2022] GET /identity/v3/role_assignments?user.id=6fdb844dddc042b9958d0532be8d63c7&scope.project.id=73b2d8f446f84ec69cdd45be83d60f38&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:07.442331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb8c04-f5db-48d0-a719-1274fa104f90 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:07.475832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c029ec-2172-4af6-b2ac-80bb7c584581 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:07.517820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179244a-47a5-4efc-b8bd-15e2487407c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:07.518530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179244a-47a5-4efc-b8bd-15e2487407c5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.518826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179244a-47a5-4efc-b8bd-15e2487407c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.519090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179244a-47a5-4efc-b8bd-15e2487407c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.539400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:07.565576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb8c04-f5db-48d0-a719-1274fa104f90 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:07.592147 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:07.601710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179244a-47a5-4efc-b8bd-15e2487407c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:07.603262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179244a-47a5-4efc-b8bd-15e2487407c5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:07.606650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a7a03-a9e5-42fb-90af-13e59a848303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:07.607685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a7a03-a9e5-42fb-90af-13e59a848303 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.607685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a7a03-a9e5-42fb-90af-13e59a848303 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.607685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a7a03-a9e5-42fb-90af-13e59a848303 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.620869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a7a03-a9e5-42fb-90af-13e59a848303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-886a7a03-a9e5-42fb-90af-13e59a848303 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:07.629182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:07.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:07.630906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 90/187] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:51:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:07.631559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a7a03-a9e5-42fb-90af-13e59a848303 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:07.713587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 98/188] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:07.714551 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179244a-47a5-4efc-b8bd-15e2487407c5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:07.738943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a7a03-a9e5-42fb-90af-13e59a848303 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:07.772941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eab0a8-84d2-4a96-b007-d6551d2d4958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:07.773773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eab0a8-84d2-4a96-b007-d6551d2d4958 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.774054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eab0a8-84d2-4a96-b007-d6551d2d4958 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.774364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eab0a8-84d2-4a96-b007-d6551d2d4958 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.781535 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:07.786181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eab0a8-84d2-4a96-b007-d6551d2d4958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f8eab0a8-84d2-4a96-b007-d6551d2d4958 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:07.794524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:07.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 91/189] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:07.796522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eab0a8-84d2-4a96-b007-d6551d2d4958 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:07.810572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:07.817503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179244a-47a5-4efc-b8bd-15e2487407c5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:07.852568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee718c98-fa75-4b48-aa8c-1c4bb8919e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:07.853371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee718c98-fa75-4b48-aa8c-1c4bb8919e6f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.853651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee718c98-fa75-4b48-aa8c-1c4bb8919e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.853919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee718c98-fa75-4b48-aa8c-1c4bb8919e6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.855235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee718c98-fa75-4b48-aa8c-1c4bb8919e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:07.856762 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee718c98-fa75-4b48-aa8c-1c4bb8919e6f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:07.858469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 99/190] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:07.859204 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee718c98-fa75-4b48-aa8c-1c4bb8919e6f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:07.896824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eab0a8-84d2-4a96-b007-d6551d2d4958 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:07.935100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab4619-c495-4faa-81fd-a16d70a39d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:07.935782 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab4619-c495-4faa-81fd-a16d70a39d89 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.936070 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab4619-c495-4faa-81fd-a16d70a39d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.936350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab4619-c495-4faa-81fd-a16d70a39d89 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.937109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab4619-c495-4faa-81fd-a16d70a39d89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:07.937425 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab4619-c495-4faa-81fd-a16d70a39d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:07.938880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab4619-c495-4faa-81fd-a16d70a39d89 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:07.946849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 92/191] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:07.949339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab4619-c495-4faa-81fd-a16d70a39d89 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:07.962257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee718c98-fa75-4b48-aa8c-1c4bb8919e6f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:07.996783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d4b4c-23e0-487a-a0ae-e157a8facb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:07.997475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d4b4c-23e0-487a-a0ae-e157a8facb9e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:07.997758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d4b4c-23e0-487a-a0ae-e157a8facb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:07.998037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d4b4c-23e0-487a-a0ae-e157a8facb9e None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:07.998844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d4b4c-23e0-487a-a0ae-e157a8facb9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:07.999183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d4b4c-23e0-487a-a0ae-e157a8facb9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:08.000545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d4b4c-23e0-487a-a0ae-e157a8facb9e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:08.008810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 100/192] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:08.009737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d4b4c-23e0-487a-a0ae-e157a8facb9e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:08.009941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:08.037740 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:08.054593 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab4619-c495-4faa-81fd-a16d70a39d89 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:08.089372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf3cc2-0c66-4e43-b7d6-b30d48cb787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:08.090351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf3cc2-0c66-4e43-b7d6-b30d48cb787a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:08.090771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf3cc2-0c66-4e43-b7d6-b30d48cb787a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:08.091052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf3cc2-0c66-4e43-b7d6-b30d48cb787a None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:08.092294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf3cc2-0c66-4e43-b7d6-b30d48cb787a None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:08.093772 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf3cc2-0c66-4e43-b7d6-b30d48cb787a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:08.095543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 93/193] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:07 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:08.096279 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf3cc2-0c66-4e43-b7d6-b30d48cb787a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:08.112120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d4b4c-23e0-487a-a0ae-e157a8facb9e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:08.145959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafae442-5018-4d49-9441-e81e6aabfb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:08.146654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafae442-5018-4d49-9441-e81e6aabfb6e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:08.146933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafae442-5018-4d49-9441-e81e6aabfb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:08.147198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafae442-5018-4d49-9441-e81e6aabfb6e None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:08.147991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafae442-5018-4d49-9441-e81e6aabfb6e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:08.148296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafae442-5018-4d49-9441-e81e6aabfb6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:08.149690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafae442-5018-4d49-9441-e81e6aabfb6e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:08.157686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 101/194] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:08.159761 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafae442-5018-4d49-9441-e81e6aabfb6e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:08.196313 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf3cc2-0c66-4e43-b7d6-b30d48cb787a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:08.221978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:08.229679 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a55158-4e28-421f-9065-8605f0b47db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:08.230385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a55158-4e28-421f-9065-8605f0b47db8 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:08.230723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a55158-4e28-421f-9065-8605f0b47db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:08.231000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a55158-4e28-421f-9065-8605f0b47db8 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:08.231801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a55158-4e28-421f-9065-8605f0b47db8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:08.233260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a55158-4e28-421f-9065-8605f0b47db8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:08.243988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:08.252802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 94/195] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:51:08 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:08.253616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a55158-4e28-421f-9065-8605f0b47db8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:08.259945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafae442-5018-4d49-9441-e81e6aabfb6e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:08.298918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635cbed-5f7c-450d-8e0c-535312d8c811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:08.299877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635cbed-5f7c-450d-8e0c-535312d8c811 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:08.300211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635cbed-5f7c-450d-8e0c-535312d8c811 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:08.300475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635cbed-5f7c-450d-8e0c-535312d8c811 None admin] PATH_INFO: `/v3/users/c90805ff76184accb2a79910a58a01d2` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:08.316561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635cbed-5f7c-450d-8e0c-535312d8c811 None admin] RBAC: Authorizing `identity:get_user(user_id=c90805ff76184accb2a79910a58a01d2)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:08.317982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635cbed-5f7c-450d-8e0c-535312d8c811 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:08.319825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 102/196] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:08 2022] GET /identity/v3/users/c90805ff76184accb2a79910a58a01d2 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:08.320646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635cbed-5f7c-450d-8e0c-535312d8c811 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:08.392795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a55158-4e28-421f-9065-8605f0b47db8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:08.432518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01070b8-1080-42ae-a6ab-13020f688be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:08.433412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01070b8-1080-42ae-a6ab-13020f688be0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:08.433700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01070b8-1080-42ae-a6ab-13020f688be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:08.434271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01070b8-1080-42ae-a6ab-13020f688be0 None admin] PATH_INFO: `/v3/users/c90805ff76184accb2a79910a58a01d2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:08.434864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635cbed-5f7c-450d-8e0c-535312d8c811 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:08.435492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01070b8-1080-42ae-a6ab-13020f688be0 None admin] RBAC: Authorizing `identity:get_user(user_id=c90805ff76184accb2a79910a58a01d2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:08.436954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01070b8-1080-42ae-a6ab-13020f688be0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:08.438650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 95/197] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:08 2022] GET /identity/v3/users/c90805ff76184accb2a79910a58a01d2 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:08.439410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01070b8-1080-42ae-a6ab-13020f688be0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:08.479954 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d93f70-c1b5-4be7-8ec3-ff23c056512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:08.480736 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d93f70-c1b5-4be7-8ec3-ff23c056512e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:08.481702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d93f70-c1b5-4be7-8ec3-ff23c056512e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:08.481702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d93f70-c1b5-4be7-8ec3-ff23c056512e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:08.483871 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d93f70-c1b5-4be7-8ec3-ff23c056512e None admin] RBAC: Adding query filter params (role.id=3e46d43e30ef494087da2c44f7ca702f, scope.project.id=9dd8fde445f5496fab61c82990f8ba09, user.id=6fdb844dddc042b9958d0532be8d63c7) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:08.483871 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d93f70-c1b5-4be7-8ec3-ff23c056512e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:08.484018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d93f70-c1b5-4be7-8ec3-ff23c056512e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:08.493094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 103/198] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:08 2022] GET /identity/v3/role_assignments?user.id=6fdb844dddc042b9958d0532be8d63c7&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:08.498125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d93f70-c1b5-4be7-8ec3-ff23c056512e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:08.505088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:08.508140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:08.580735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01070b8-1080-42ae-a6ab-13020f688be0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:08.624738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa3b92-93bb-43bf-99c6-b4a26ba5c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:08.625882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa3b92-93bb-43bf-99c6-b4a26ba5c5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:08.626337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa3b92-93bb-43bf-99c6-b4a26ba5c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:08.626750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa3b92-93bb-43bf-99c6-b4a26ba5c5d4 None admin] PATH_INFO: `/v3/users/c4793dfbf0514b038e9957c59546b023` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:08.628187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa3b92-93bb-43bf-99c6-b4a26ba5c5d4 None admin] RBAC: Authorizing `identity:get_user(user_id=c4793dfbf0514b038e9957c59546b023)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:08.629835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa3b92-93bb-43bf-99c6-b4a26ba5c5d4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:08.631761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 96/199] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:08 2022] GET /identity/v3/users/c4793dfbf0514b038e9957c59546b023 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:08.634402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d93f70-c1b5-4be7-8ec3-ff23c056512e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:08.635199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa3b92-93bb-43bf-99c6-b4a26ba5c5d4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:08.674538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fcd59-375a-4c36-8c53-c817b7fa6c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:08.675224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fcd59-375a-4c36-8c53-c817b7fa6c1b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:08.675506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fcd59-375a-4c36-8c53-c817b7fa6c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:08.675785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fcd59-375a-4c36-8c53-c817b7fa6c1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:08.765939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fcd59-375a-4c36-8c53-c817b7fa6c1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:08.768080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fcd59-375a-4c36-8c53-c817b7fa6c1b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:08.776600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa3b92-93bb-43bf-99c6-b4a26ba5c5d4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:08.782885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:08.800658 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:08.817431 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3521ad-a3f5-4342-89e6-3eb24eb6d4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:08.818238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3521ad-a3f5-4342-89e6-3eb24eb6d4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:08.818591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3521ad-a3f5-4342-89e6-3eb24eb6d4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:08.818879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3521ad-a3f5-4342-89e6-3eb24eb6d4e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:08.894983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3521ad-a3f5-4342-89e6-3eb24eb6d4e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:08.896580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3521ad-a3f5-4342-89e6-3eb24eb6d4e9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:08.901431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 104/200] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:08.902068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fcd59-375a-4c36-8c53-c817b7fa6c1b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:09.021730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fcd59-375a-4c36-8c53-c817b7fa6c1b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:09.025999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 97/201] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:09.026898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3521ad-a3f5-4342-89e6-3eb24eb6d4e9 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:09.032392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:09.058523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:09.065027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3bd55-fe39-4ad1-9b37-30b210a04fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:09.066646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3bd55-fe39-4ad1-9b37-30b210a04fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:09.066984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3bd55-fe39-4ad1-9b37-30b210a04fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:09.067285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3bd55-fe39-4ad1-9b37-30b210a04fb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:09.146629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3bd55-fe39-4ad1-9b37-30b210a04fb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:09.148025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3bd55-fe39-4ad1-9b37-30b210a04fb6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:09.156710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3521ad-a3f5-4342-89e6-3eb24eb6d4e9 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:09.205643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f007688-2286-42dd-8f09-261776c2c2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:09.206822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f007688-2286-42dd-8f09-261776c2c2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:09.207339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f007688-2286-42dd-8f09-261776c2c2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:09.207624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f007688-2286-42dd-8f09-261776c2c2d8 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:09.209945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f007688-2286-42dd-8f09-261776c2c2d8 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:09.212110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f007688-2286-42dd-8f09-261776c2c2d8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:09.214147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 98/202] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:09 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:09.214935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f007688-2286-42dd-8f09-261776c2c2d8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:09.259022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 105/203] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:09.259668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3bd55-fe39-4ad1-9b37-30b210a04fb6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:09.271189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:09.338736 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:09.356733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f007688-2286-42dd-8f09-261776c2c2d8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:09.371628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3bd55-fe39-4ad1-9b37-30b210a04fb6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:09.394286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d90264-0232-4575-b299-d8c0bc1c6e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:09.395163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d90264-0232-4575-b299-d8c0bc1c6e13 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:09.395442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d90264-0232-4575-b299-d8c0bc1c6e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:09.395711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d90264-0232-4575-b299-d8c0bc1c6e13 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:09.396947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d90264-0232-4575-b299-d8c0bc1c6e13 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:09.398362 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d90264-0232-4575-b299-d8c0bc1c6e13 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:09.400391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 99/204] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:09 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:09.401201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d90264-0232-4575-b299-d8c0bc1c6e13 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:09.414993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bc15f-422a-44d8-804f-dc5db2fe866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:09.415873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bc15f-422a-44d8-804f-dc5db2fe866b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:09.416450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bc15f-422a-44d8-804f-dc5db2fe866b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:09.416873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bc15f-422a-44d8-804f-dc5db2fe866b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:09.418368 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bc15f-422a-44d8-804f-dc5db2fe866b None admin] RBAC: Adding query filter params (role.id=ffab93457c0f4f64a29764f17759396e, scope.project.id=205237c47ce44b8b839b43198fb4c1f1, user.id=c90805ff76184accb2a79910a58a01d2) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:09.418860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bc15f-422a-44d8-804f-dc5db2fe866b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:09.421400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bc15f-422a-44d8-804f-dc5db2fe866b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:09.429055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 106/205] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:09 2022] GET /identity/v3/role_assignments?user.id=c90805ff76184accb2a79910a58a01d2&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=ffab93457c0f4f64a29764f17759396e => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:09.430003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bc15f-422a-44d8-804f-dc5db2fe866b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:09.538076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:09.571774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bc15f-422a-44d8-804f-dc5db2fe866b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:09.575310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d90264-0232-4575-b299-d8c0bc1c6e13 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:09.608798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7200a9-d219-4d77-9df7-ac106630dc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:09.609625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7200a9-d219-4d77-9df7-ac106630dc03 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:09.610016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7200a9-d219-4d77-9df7-ac106630dc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:09.610889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7200a9-d219-4d77-9df7-ac106630dc03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:09.611990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7200a9-d219-4d77-9df7-ac106630dc03 None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=205237c47ce44b8b839b43198fb4c1f1, user.id=c90805ff76184accb2a79910a58a01d2) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:09.612420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7200a9-d219-4d77-9df7-ac106630dc03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:09.614021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7200a9-d219-4d77-9df7-ac106630dc03 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:09.621860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 107/206] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:09 2022] GET /identity/v3/role_assignments?user.id=c90805ff76184accb2a79910a58a01d2&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:09.635329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f7e0d-9244-4614-bf39-41f6bbf85838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:09.636242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f7e0d-9244-4614-bf39-41f6bbf85838 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:09.636531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f7e0d-9244-4614-bf39-41f6bbf85838 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:09.636862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f7e0d-9244-4614-bf39-41f6bbf85838 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:09.638528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f7e0d-9244-4614-bf39-41f6bbf85838 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:09.640072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f7e0d-9244-4614-bf39-41f6bbf85838 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:09.641933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 100/207] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:09 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:09.648405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7200a9-d219-4d77-9df7-ac106630dc03 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:09.762540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7200a9-d219-4d77-9df7-ac106630dc03 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:09.798671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43eb34-e1d3-4b05-816c-e1d2b654e915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:09.799354 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43eb34-e1d3-4b05-816c-e1d2b654e915 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:09.799645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43eb34-e1d3-4b05-816c-e1d2b654e915 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:09.799918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43eb34-e1d3-4b05-816c-e1d2b654e915 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:09.800764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43eb34-e1d3-4b05-816c-e1d2b654e915 None admin] RBAC: Adding query filter params (role.id=e4214ef8456541528194f90e229b356a, scope.project.id=9dd8fde445f5496fab61c82990f8ba09, user.id=c4793dfbf0514b038e9957c59546b023) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:09.801078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43eb34-e1d3-4b05-816c-e1d2b654e915 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:09.802628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43eb34-e1d3-4b05-816c-e1d2b654e915 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:09.806172 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:09.814548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 108/208] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:09 2022] GET /identity/v3/role_assignments?user.id=c4793dfbf0514b038e9957c59546b023&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=e4214ef8456541528194f90e229b356a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:10.701282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9400b9-f78e-4a19-97ef-5f33f70100be None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:10.701838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1ed16-dd54-4623-822e-5c67a618f2b6 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:10.702199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9400b9-f78e-4a19-97ef-5f33f70100be None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:10.702270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1ed16-dd54-4623-822e-5c67a618f2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:10.702348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9400b9-f78e-4a19-97ef-5f33f70100be None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:10.702416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1ed16-dd54-4623-822e-5c67a618f2b6 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:10.703394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 101/209] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:10.703492 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 109/210] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:51:10.708896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acf10f-5b1e-467a-84e8-a5bf5b48a418 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:10.709049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311e5c2-b50d-4675-acc6-f253411ca11e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:10.709218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acf10f-5b1e-467a-84e8-a5bf5b48a418 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:10.709309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311e5c2-b50d-4675-acc6-f253411ca11e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:10.709461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311e5c2-b50d-4675-acc6-f253411ca11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:10.709546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23acf10f-5b1e-467a-84e8-a5bf5b48a418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:10.728226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-23acf10f-5b1e-467a-84e8-a5bf5b48a418 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:10.729482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-23acf10f-5b1e-467a-84e8-a5bf5b48a418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:10.740362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7311e5c2-b50d-4675-acc6-f253411ca11e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:10.741953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7311e5c2-b50d-4675-acc6-f253411ca11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:10.752738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:10.816180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 102/211] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:10.820910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 110/212] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:10.841080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acf10f-5b1e-467a-84e8-a5bf5b48a418 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:10.847341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311e5c2-b50d-4675-acc6-f253411ca11e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:10.954261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23acf10f-5b1e-467a-84e8-a5bf5b48a418 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:10.977419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311e5c2-b50d-4675-acc6-f253411ca11e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:11.000679 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b178fb-40f0-461d-b08d-e1f2060f2f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:11.001639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b178fb-40f0-461d-b08d-e1f2060f2f1d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:11.002015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b178fb-40f0-461d-b08d-e1f2060f2f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:11.002471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b178fb-40f0-461d-b08d-e1f2060f2f1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:11.014015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:11.018925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b178fb-40f0-461d-b08d-e1f2060f2f1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:11.020865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d01b1f-d0b0-4aa2-a47c-7462260e154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:11.021918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d01b1f-d0b0-4aa2-a47c-7462260e154b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:11.022367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d01b1f-d0b0-4aa2-a47c-7462260e154b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:11.022780 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d01b1f-d0b0-4aa2-a47c-7462260e154b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:11.024085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d01b1f-d0b0-4aa2-a47c-7462260e154b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:11.025708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d01b1f-d0b0-4aa2-a47c-7462260e154b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-07b178fb-40f0-461d-b08d-e1f2060f2f1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:11.032054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:11.033082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:11.033874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 103/214] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:11.034551 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 111/214] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:11.039091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b178fb-40f0-461d-b08d-e1f2060f2f1d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:11.040377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d01b1f-d0b0-4aa2-a47c-7462260e154b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:11.056581 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:11.143135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b178fb-40f0-461d-b08d-e1f2060f2f1d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:11.144804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d01b1f-d0b0-4aa2-a47c-7462260e154b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:11.178603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bde02-9c94-46c1-ae91-6a7ac2d4b177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:11.178890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344a1d5-7032-485f-965b-999cdb6bbc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:11.179523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bde02-9c94-46c1-ae91-6a7ac2d4b177 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:11.179858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344a1d5-7032-485f-965b-999cdb6bbc0c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:11.179964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bde02-9c94-46c1-ae91-6a7ac2d4b177 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:11.180334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bde02-9c94-46c1-ae91-6a7ac2d4b177 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:11.180506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344a1d5-7032-485f-965b-999cdb6bbc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:11.180902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344a1d5-7032-485f-965b-999cdb6bbc0c None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:11.181335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bde02-9c94-46c1-ae91-6a7ac2d4b177 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:11.181935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344a1d5-7032-485f-965b-999cdb6bbc0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:11.182392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344a1d5-7032-485f-965b-999cdb6bbc0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:11.183104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bde02-9c94-46c1-ae91-6a7ac2d4b177 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:11.184161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344a1d5-7032-485f-965b-999cdb6bbc0c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:11.197961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 112/215] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:11.198461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344a1d5-7032-485f-965b-999cdb6bbc0c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:11.222202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 104/216] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:51:11 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:11.283518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:11.330934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344a1d5-7032-485f-965b-999cdb6bbc0c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:11.373427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c524c-775f-489b-9b5b-f3a004c6c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:11.374712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c524c-775f-489b-9b5b-f3a004c6c293 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:11.375183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c524c-775f-489b-9b5b-f3a004c6c293 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:11.375759 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c524c-775f-489b-9b5b-f3a004c6c293 None admin] PATH_INFO: `/v3/users/6e32fe77ebc04e8ca85f4c322eae083e` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:11.391154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c524c-775f-489b-9b5b-f3a004c6c293 None admin] RBAC: Authorizing `identity:get_user(user_id=6e32fe77ebc04e8ca85f4c322eae083e)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:11.392991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c524c-775f-489b-9b5b-f3a004c6c293 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:11.394980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 113/217] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:11 2022] GET /identity/v3/users/6e32fe77ebc04e8ca85f4c322eae083e => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:11.400844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bde02-9c94-46c1-ae91-6a7ac2d4b177 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:11.484920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:11.518597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bde02-9c94-46c1-ae91-6a7ac2d4b177 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:11.553465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cc3a9-b89c-4d11-97cb-1bf7bc69eeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:11.554225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cc3a9-b89c-4d11-97cb-1bf7bc69eeef None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:11.554423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cc3a9-b89c-4d11-97cb-1bf7bc69eeef None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:11.554703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cc3a9-b89c-4d11-97cb-1bf7bc69eeef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:11.629180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cc3a9-b89c-4d11-97cb-1bf7bc69eeef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:11.630725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cc3a9-b89c-4d11-97cb-1bf7bc69eeef None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:11.706485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35865e14-f07b-4725-8317-9b7cc6efed61 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:11.706995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:11.707333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35865e14-f07b-4725-8317-9b7cc6efed61 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:11.707333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35865e14-f07b-4725-8317-9b7cc6efed61 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:11.709728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 114/218] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:51:11.715860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13112a35-76bb-49e7-a0cd-dc06d05d2b60 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:11.716363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13112a35-76bb-49e7-a0cd-dc06d05d2b60 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:11.716802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13112a35-76bb-49e7-a0cd-dc06d05d2b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:11.739662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-13112a35-76bb-49e7-a0cd-dc06d05d2b60 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:11.741129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-13112a35-76bb-49e7-a0cd-dc06d05d2b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:11.744025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 105/219] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:11.750510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cc3a9-b89c-4d11-97cb-1bf7bc69eeef None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:11.822595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 115/220] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:11.838334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13112a35-76bb-49e7-a0cd-dc06d05d2b60 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:11.858791 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cc3a9-b89c-4d11-97cb-1bf7bc69eeef None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:11.895496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb556d-75bc-46a9-885b-f1e22572106a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:11.896757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb556d-75bc-46a9-885b-f1e22572106a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:11.897207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb556d-75bc-46a9-885b-f1e22572106a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:11.897673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb556d-75bc-46a9-885b-f1e22572106a None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:11.899159 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb556d-75bc-46a9-885b-f1e22572106a None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:11.900886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb556d-75bc-46a9-885b-f1e22572106a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:11.904008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 106/221] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:11 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:11.913693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb556d-75bc-46a9-885b-f1e22572106a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:11.934727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:11.966502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:11.980162 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13112a35-76bb-49e7-a0cd-dc06d05d2b60 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:12.015320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c23c6-61b3-423c-ab63-c8320aabcace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:12.016085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c23c6-61b3-423c-ab63-c8320aabcace None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:12.016361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c23c6-61b3-423c-ab63-c8320aabcace None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:12.016630 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c23c6-61b3-423c-ab63-c8320aabcace None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:12.026964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c23c6-61b3-423c-ab63-c8320aabcace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-219c23c6-61b3-423c-ab63-c8320aabcace None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:12.035308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:12.036207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:12.036987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:12.040788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb556d-75bc-46a9-885b-f1e22572106a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:12.041823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 116/222] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:12.043433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c6ed4-ead1-4f08-a8e2-231b02b5c2c4 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:12.044418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c6ed4-ead1-4f08-a8e2-231b02b5c2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:12.044418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c6ed4-ead1-4f08-a8e2-231b02b5c2c4 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:12.045962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 117/223] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:12.045962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c6ed4-ead1-4f08-a8e2-231b02b5c2c4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:12.091079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c708aa-7d13-4913-b394-c7e49f9a1b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:12.091741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c708aa-7d13-4913-b394-c7e49f9a1b59 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:12.092018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c708aa-7d13-4913-b394-c7e49f9a1b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:12.092286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c708aa-7d13-4913-b394-c7e49f9a1b59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:12.093117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c708aa-7d13-4913-b394-c7e49f9a1b59 None admin] RBAC: Adding query filter params (role.id=3e46d43e30ef494087da2c44f7ca702f, scope.project.id=205237c47ce44b8b839b43198fb4c1f1, user.id=6e32fe77ebc04e8ca85f4c322eae083e) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:12.093424 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c708aa-7d13-4913-b394-c7e49f9a1b59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:12.094914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c708aa-7d13-4913-b394-c7e49f9a1b59 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:12.100992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 107/224] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:11 2022] GET /identity/v3/role_assignments?user.id=6e32fe77ebc04e8ca85f4c322eae083e&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:12.102048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3370f23-ff08-4a03-8066-d0a1bba0971f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:12.102379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3370f23-ff08-4a03-8066-d0a1bba0971f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:12.102657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3370f23-ff08-4a03-8066-d0a1bba0971f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:12.120815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d3370f23-ff08-4a03-8066-d0a1bba0971f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:12.122468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d3370f23-ff08-4a03-8066-d0a1bba0971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:12.200199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c6ed4-ead1-4f08-a8e2-231b02b5c2c4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:12.214715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:12.218233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 108/225] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:12.241531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3370f23-ff08-4a03-8066-d0a1bba0971f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:12.274112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9792f79-ea36-415f-8834-60c9eecd281a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:12.275573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9792f79-ea36-415f-8834-60c9eecd281a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:12.276078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9792f79-ea36-415f-8834-60c9eecd281a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:12.276363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9792f79-ea36-415f-8834-60c9eecd281a None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:12.277151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9792f79-ea36-415f-8834-60c9eecd281a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:12.277894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9792f79-ea36-415f-8834-60c9eecd281a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:12.280888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9792f79-ea36-415f-8834-60c9eecd281a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:12.287221 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:12.301913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 118/226] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:12.307334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9792f79-ea36-415f-8834-60c9eecd281a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:12.386454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3370f23-ff08-4a03-8066-d0a1bba0971f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:12.442418 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e4617-0a81-4792-ab57-023a4526d339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:12.443240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e4617-0a81-4792-ab57-023a4526d339 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:12.443521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e4617-0a81-4792-ab57-023a4526d339 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:12.443792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e4617-0a81-4792-ab57-023a4526d339 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:12.464957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e4617-0a81-4792-ab57-023a4526d339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-730e4617-0a81-4792-ab57-023a4526d339 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:12.477836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:12.478763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:12.479542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:12.491062 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 109/227] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:51:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:12.497876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9792f79-ea36-415f-8834-60c9eecd281a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:12.501907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e4617-0a81-4792-ab57-023a4526d339 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:12.544854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37519417-8eb2-404b-be1c-4c7271dc613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:12.545738 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37519417-8eb2-404b-be1c-4c7271dc613e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:12.546021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37519417-8eb2-404b-be1c-4c7271dc613e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:12.546324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37519417-8eb2-404b-be1c-4c7271dc613e None admin] PATH_INFO: `/v3/users/c90805ff76184accb2a79910a58a01d2` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:12.547646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37519417-8eb2-404b-be1c-4c7271dc613e None admin] RBAC: Authorizing `identity:get_user(user_id=c90805ff76184accb2a79910a58a01d2)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:12.549184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37519417-8eb2-404b-be1c-4c7271dc613e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:12.555374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 119/228] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:12 2022] GET /identity/v3/users/c90805ff76184accb2a79910a58a01d2 => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:12.561222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37519417-8eb2-404b-be1c-4c7271dc613e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:12.642493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:12.650108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e4617-0a81-4792-ab57-023a4526d339 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:12.650246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:12.709214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d1a3a-72d6-4e9b-8c63-446c60eea2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:12.709898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d1a3a-72d6-4e9b-8c63-446c60eea2fe None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:12.710196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d1a3a-72d6-4e9b-8c63-446c60eea2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:12.710465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d1a3a-72d6-4e9b-8c63-446c60eea2fe None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:12.711320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d1a3a-72d6-4e9b-8c63-446c60eea2fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:12.711630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d1a3a-72d6-4e9b-8c63-446c60eea2fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:12.715828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d1a3a-72d6-4e9b-8c63-446c60eea2fe None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:12.729247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 110/229] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:12.736398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d1a3a-72d6-4e9b-8c63-446c60eea2fe None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:12.774856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37519417-8eb2-404b-be1c-4c7271dc613e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:12.842711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc38aaf-5193-41b8-ab67-3886abcde0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:12.843510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc38aaf-5193-41b8-ab67-3886abcde0fc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:12.843791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc38aaf-5193-41b8-ab67-3886abcde0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:12.844069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc38aaf-5193-41b8-ab67-3886abcde0fc None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:12.845970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc38aaf-5193-41b8-ab67-3886abcde0fc None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:12.847476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc38aaf-5193-41b8-ab67-3886abcde0fc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:12.850367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 120/230] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:12 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:12.854605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc38aaf-5193-41b8-ab67-3886abcde0fc None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:12.968335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d1a3a-72d6-4e9b-8c63-446c60eea2fe None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:12.985682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc38aaf-5193-41b8-ab67-3886abcde0fc None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:12.993531 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:13.036720 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594eccbf-e3d9-416c-a00a-cf869920da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:13.038158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eccbf-e3d9-416c-a00a-cf869920da49 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:13.038443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eccbf-e3d9-416c-a00a-cf869920da49 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:13.038726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594eccbf-e3d9-416c-a00a-cf869920da49 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1/users/c90805ff76184accb2a79910a58a01d2/roles/ffab93457c0f4f64a29764f17759396e` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:13.042939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1174ad-17b9-4cf4-ac6b-36ecdea56056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:13.044890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594eccbf-e3d9-416c-a00a-cf869920da49 None admin] RBAC: Authorizing `identity:create_grant(project_id=205237c47ce44b8b839b43198fb4c1f1, user_id=c90805ff76184accb2a79910a58a01d2, role_id=ffab93457c0f4f64a29764f17759396e)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:13.045927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1174ad-17b9-4cf4-ac6b-36ecdea56056 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:13.046244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1174ad-17b9-4cf4-ac6b-36ecdea56056 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:13.046520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1174ad-17b9-4cf4-ac6b-36ecdea56056 None admin] PATH_INFO: `/v3/users/c90805ff76184accb2a79910a58a01d2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:13.047740 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594eccbf-e3d9-416c-a00a-cf869920da49 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:13.049321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1174ad-17b9-4cf4-ac6b-36ecdea56056 None admin] RBAC: Authorizing `identity:get_user(user_id=c90805ff76184accb2a79910a58a01d2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:13.052393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1174ad-17b9-4cf4-ac6b-36ecdea56056 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:13.056212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 111/231] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:12 2022] GET /identity/v3/users/c90805ff76184accb2a79910a58a01d2 => generated 283 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:13.063299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1174ad-17b9-4cf4-ac6b-36ecdea56056 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:13.087100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 121/232] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:12 2022] PUT /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1/users/c90805ff76184accb2a79910a58a01d2/roles/ffab93457c0f4f64a29764f17759396e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:13.106606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:13.208068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1174ad-17b9-4cf4-ac6b-36ecdea56056 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:13.257690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67d484-b277-447b-9cac-d5a840d44110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:13.258600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67d484-b277-447b-9cac-d5a840d44110 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:13.258886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67d484-b277-447b-9cac-d5a840d44110 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:13.259165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67d484-b277-447b-9cac-d5a840d44110 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:13.260357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67d484-b277-447b-9cac-d5a840d44110 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:13.261865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67d484-b277-447b-9cac-d5a840d44110 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:13.268446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 112/233] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:13 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:13.269868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594eccbf-e3d9-416c-a00a-cf869920da49 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:13.378080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594eccbf-e3d9-416c-a00a-cf869920da49 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:13.412756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e73406-b62d-4365-93cb-6541814d8a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:13.414021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e73406-b62d-4365-93cb-6541814d8a5d None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:13.414494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e73406-b62d-4365-93cb-6541814d8a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:13.415067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e73406-b62d-4365-93cb-6541814d8a5d None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1/users/c90805ff76184accb2a79910a58a01d2/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:13.417347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e73406-b62d-4365-93cb-6541814d8a5d None admin] RBAC: Authorizing `identity:create_grant(project_id=205237c47ce44b8b839b43198fb4c1f1, user_id=c90805ff76184accb2a79910a58a01d2, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:13.419159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e73406-b62d-4365-93cb-6541814d8a5d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:13.447329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 122/234] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:13 2022] PUT /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1/users/c90805ff76184accb2a79910a58a01d2/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:13.827965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237882b-507d-447c-ac33-8042171068a9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:13.828946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237882b-507d-447c-ac33-8042171068a9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:13.829397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237882b-507d-447c-ac33-8042171068a9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:13.831250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 113/235] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:13.837454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13963879-158f-4264-8b30-d947a68ee6db None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:13.838272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13963879-158f-4264-8b30-d947a68ee6db None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:13.838745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13963879-158f-4264-8b30-d947a68ee6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:13.860367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-13963879-158f-4264-8b30-d947a68ee6db None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:13.861828 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-13963879-158f-4264-8b30-d947a68ee6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:13.888017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:13.945501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 123/236] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:13.968765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c0cc2-7195-49ae-97c2-0b78e5115677 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:13.969051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c0cc2-7195-49ae-97c2-0b78e5115677 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:13.969334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c0cc2-7195-49ae-97c2-0b78e5115677 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:13.970914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 114/237] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:13.975957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4327a90-3996-43ba-b6e7-9b19b2af8ae5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:13.976249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4327a90-3996-43ba-b6e7-9b19b2af8ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:13.976524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4327a90-3996-43ba-b6e7-9b19b2af8ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:13.997880 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a4327a90-3996-43ba-b6e7-9b19b2af8ae5 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:13.999159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a4327a90-3996-43ba-b6e7-9b19b2af8ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:14.025526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c0cc2-7195-49ae-97c2-0b78e5115677 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:14.121347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 124/238] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:14.153806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4327a90-3996-43ba-b6e7-9b19b2af8ae5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:14.168943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:14.180276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c0cc2-7195-49ae-97c2-0b78e5115677 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:14.240879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d052d-ea37-4c79-ac7d-c76fb3c285e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:14.242126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d052d-ea37-4c79-ac7d-c76fb3c285e3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:14.242602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d052d-ea37-4c79-ac7d-c76fb3c285e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:14.243077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d052d-ea37-4c79-ac7d-c76fb3c285e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:14.263832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d052d-ea37-4c79-ac7d-c76fb3c285e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f62d052d-ea37-4c79-ac7d-c76fb3c285e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:14.278967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:14.279876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:14.280649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:14.283630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 115/239] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:14.289320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d052d-ea37-4c79-ac7d-c76fb3c285e3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:14.343851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:14.374688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4327a90-3996-43ba-b6e7-9b19b2af8ae5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:14.419731 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d052d-ea37-4c79-ac7d-c76fb3c285e3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:14.460496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e7e2-5ead-4fe4-bd1b-fdfcb2a830e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:14.462636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e7e2-5ead-4fe4-bd1b-fdfcb2a830e8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:14.464312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e7e2-5ead-4fe4-bd1b-fdfcb2a830e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:14.466102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e7e2-5ead-4fe4-bd1b-fdfcb2a830e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:14.468180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013e7e2-5ead-4fe4-bd1b-fdfcb2a830e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:14.470105 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013e7e2-5ead-4fe4-bd1b-fdfcb2a830e8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:14.473565 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 125/240] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:14.476137 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3878d-331d-460d-b901-dbd3b37017d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:14.477042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3878d-331d-460d-b901-dbd3b37017d8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:14.477992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e7e2-5ead-4fe4-bd1b-fdfcb2a830e8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:14.478856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3878d-331d-460d-b901-dbd3b37017d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:14.479653 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3878d-331d-460d-b901-dbd3b37017d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:14.482579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3878d-331d-460d-b901-dbd3b37017d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:14.483109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3878d-331d-460d-b901-dbd3b37017d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:14.484703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3878d-331d-460d-b901-dbd3b37017d8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:14.504025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 116/241] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:14.510152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3878d-331d-460d-b901-dbd3b37017d8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:14.538678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:14.647612 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3878d-331d-460d-b901-dbd3b37017d8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:14.648389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e7e2-5ead-4fe4-bd1b-fdfcb2a830e8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:14.683527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ab75f-a8f8-4b33-852a-0e373abd0b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:14.684242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ab75f-a8f8-4b33-852a-0e373abd0b4b None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:14.684520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ab75f-a8f8-4b33-852a-0e373abd0b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:14.684685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e84626-c795-4cfe-a899-3a9afb9ef299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:14.684793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ab75f-a8f8-4b33-852a-0e373abd0b4b None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:14.685503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ab75f-a8f8-4b33-852a-0e373abd0b4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:14.685602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84626-c795-4cfe-a899-3a9afb9ef299 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:14.685831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84626-c795-4cfe-a899-3a9afb9ef299 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:14.686125 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e84626-c795-4cfe-a899-3a9afb9ef299 None admin] PATH_INFO: `/v3/users/dce50a7df665449baa3e629e32f6ef75` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:14.686988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ab75f-a8f8-4b33-852a-0e373abd0b4b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:14.700898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e84626-c795-4cfe-a899-3a9afb9ef299 None admin] RBAC: Authorizing `identity:get_user(user_id=dce50a7df665449baa3e629e32f6ef75)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:14.702475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e84626-c795-4cfe-a899-3a9afb9ef299 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:14.704476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 117/242] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:14 2022] GET /identity/v3/users/dce50a7df665449baa3e629e32f6ef75 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:14.705855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0139b-80cc-49ef-8c24-5874c7bcef94 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:14.710269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 126/243] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:51:14 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:14.710492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0139b-80cc-49ef-8c24-5874c7bcef94 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:14.710826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0139b-80cc-49ef-8c24-5874c7bcef94 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:14.711143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ab75f-a8f8-4b33-852a-0e373abd0b4b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:14.712535 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 118/244] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:14.721888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e8c6b-7cdd-4398-97ca-83053ea9c285 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:14.722290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e8c6b-7cdd-4398-97ca-83053ea9c285 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:14.722474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e8c6b-7cdd-4398-97ca-83053ea9c285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:14.744687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:14.755502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-094e8c6b-7cdd-4398-97ca-83053ea9c285 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:14.756867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-094e8c6b-7cdd-4398-97ca-83053ea9c285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:14.878620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:14.887982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 119/245] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:14.916784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e8c6b-7cdd-4398-97ca-83053ea9c285 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:14.929267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ab75f-a8f8-4b33-852a-0e373abd0b4b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:14.969160 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcacaa5-f4d1-40cb-92fa-c662ff146564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:14.969829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcacaa5-f4d1-40cb-92fa-c662ff146564 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:14.970147 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcacaa5-f4d1-40cb-92fa-c662ff146564 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:14.970445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcacaa5-f4d1-40cb-92fa-c662ff146564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.036367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e8c6b-7cdd-4398-97ca-83053ea9c285 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:15.056801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcacaa5-f4d1-40cb-92fa-c662ff146564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:15.059074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcacaa5-f4d1-40cb-92fa-c662ff146564 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:15.074506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:15.076547 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c777bfe-2bfa-4129-aad3-b9e05eb498e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:15.077400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c777bfe-2bfa-4129-aad3-b9e05eb498e1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:15.077682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c777bfe-2bfa-4129-aad3-b9e05eb498e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:15.077956 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c777bfe-2bfa-4129-aad3-b9e05eb498e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.093125 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c777bfe-2bfa-4129-aad3-b9e05eb498e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-9c777bfe-2bfa-4129-aad3-b9e05eb498e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:15.101582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:15.102606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:15.103398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:15.109237 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 120/246] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:15.110788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c777bfe-2bfa-4129-aad3-b9e05eb498e1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:15.182542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 127/247] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:15.189309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcacaa5-f4d1-40cb-92fa-c662ff146564 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:15.191200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:15.247440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c777bfe-2bfa-4129-aad3-b9e05eb498e1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:15.291646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0a054-eab0-4d53-b762-332ffef1715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:15.292657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0a054-eab0-4d53-b762-332ffef1715d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:15.293173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0a054-eab0-4d53-b762-332ffef1715d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:15.293620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0a054-eab0-4d53-b762-332ffef1715d None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.294746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0a054-eab0-4d53-b762-332ffef1715d None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:15.295233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0a054-eab0-4d53-b762-332ffef1715d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:15.296840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0a054-eab0-4d53-b762-332ffef1715d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:15.308308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 121/248] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:15.310151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcacaa5-f4d1-40cb-92fa-c662ff146564 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:15.315306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0a054-eab0-4d53-b762-332ffef1715d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:15.322419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:15.355645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea278f-f7e7-453c-a4fb-09cdc8d493c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:15.356629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea278f-f7e7-453c-a4fb-09cdc8d493c9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:15.356914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea278f-f7e7-453c-a4fb-09cdc8d493c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:15.357190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea278f-f7e7-453c-a4fb-09cdc8d493c9 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.360513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea278f-f7e7-453c-a4fb-09cdc8d493c9 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:15.362007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea278f-f7e7-453c-a4fb-09cdc8d493c9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:15.368832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 128/249] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:15 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:15.375114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea278f-f7e7-453c-a4fb-09cdc8d493c9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:15.445000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0a054-eab0-4d53-b762-332ffef1715d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:15.474764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:15.489563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdee560-bafb-4073-b95c-3836535f08c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:15.491014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdee560-bafb-4073-b95c-3836535f08c0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:15.491492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdee560-bafb-4073-b95c-3836535f08c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:15.492039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdee560-bafb-4073-b95c-3836535f08c0 None admin] PATH_INFO: `/v3/users/6e32fe77ebc04e8ca85f4c322eae083e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.493611 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdee560-bafb-4073-b95c-3836535f08c0 None admin] RBAC: Authorizing `identity:get_user(user_id=6e32fe77ebc04e8ca85f4c322eae083e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:15.496940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdee560-bafb-4073-b95c-3836535f08c0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:15.499053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 122/250] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:15 2022] GET /identity/v3/users/6e32fe77ebc04e8ca85f4c322eae083e => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:15.501677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea278f-f7e7-453c-a4fb-09cdc8d493c9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:15.505211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdee560-bafb-4073-b95c-3836535f08c0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:15.540482 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e9ea-a6e9-44e5-b5f9-cc9781fadc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:15.541154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e9ea-a6e9-44e5-b5f9-cc9781fadc6d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:15.541444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e9ea-a6e9-44e5-b5f9-cc9781fadc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:15.541861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412e9ea-a6e9-44e5-b5f9-cc9781fadc6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.543084 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412e9ea-a6e9-44e5-b5f9-cc9781fadc6d None admin] RBAC: Adding query filter params (role.id=e4214ef8456541528194f90e229b356a, scope.project.id=205237c47ce44b8b839b43198fb4c1f1, user.id=dce50a7df665449baa3e629e32f6ef75) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:15.543406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412e9ea-a6e9-44e5-b5f9-cc9781fadc6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:15.545194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412e9ea-a6e9-44e5-b5f9-cc9781fadc6d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:15.552026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 129/251] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:15 2022] GET /identity/v3/role_assignments?user.id=dce50a7df665449baa3e629e32f6ef75&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=e4214ef8456541528194f90e229b356a => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:15.630959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdee560-bafb-4073-b95c-3836535f08c0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:15.674068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9195bb-af5c-4b67-ab4f-ba1bc32714f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:15.675006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9195bb-af5c-4b67-ab4f-ba1bc32714f7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:15.675290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9195bb-af5c-4b67-ab4f-ba1bc32714f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:15.675570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9195bb-af5c-4b67-ab4f-ba1bc32714f7 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.676813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9195bb-af5c-4b67-ab4f-ba1bc32714f7 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:15.678382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9195bb-af5c-4b67-ab4f-ba1bc32714f7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:15.680178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 123/252] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:15 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:15.686210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e9ea-a6e9-44e5-b5f9-cc9781fadc6d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:15.736408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:15.794563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412e9ea-a6e9-44e5-b5f9-cc9781fadc6d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:15.814937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86662f2a-cfda-4c86-8dfa-7a2cfe58b337 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:15.815247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86662f2a-cfda-4c86-8dfa-7a2cfe58b337 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:15.815496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86662f2a-cfda-4c86-8dfa-7a2cfe58b337 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.817064 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 124/253] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:15.835646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c239cc3-2e61-4080-bdfd-5aef3ff1e0f5 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:15.835956 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c239cc3-2e61-4080-bdfd-5aef3ff1e0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:15.836208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c239cc3-2e61-4080-bdfd-5aef3ff1e0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.857533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c01fb-1850-40de-b51b-9fee685e6c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:15.862587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c01fb-1850-40de-b51b-9fee685e6c31 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:15.862882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c01fb-1850-40de-b51b-9fee685e6c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:15.863166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c01fb-1850-40de-b51b-9fee685e6c31 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1/users/6e32fe77ebc04e8ca85f4c322eae083e/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.866118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c01fb-1850-40de-b51b-9fee685e6c31 None admin] RBAC: Authorizing `identity:create_grant(project_id=205237c47ce44b8b839b43198fb4c1f1, user_id=6e32fe77ebc04e8ca85f4c322eae083e, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:15.867169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-2c239cc3-2e61-4080-bdfd-5aef3ff1e0f5 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:15.867599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c01fb-1850-40de-b51b-9fee685e6c31 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:15.868384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2c239cc3-2e61-4080-bdfd-5aef3ff1e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:15.895253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 130/254] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:15 2022] PUT /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1/users/6e32fe77ebc04e8ca85f4c322eae083e/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:15.974113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:15.983479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0546f98-629f-4ad1-8b3c-fdfe4c6cb404 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:15.983973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0546f98-629f-4ad1-8b3c-fdfe4c6cb404 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:15.984045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0546f98-629f-4ad1-8b3c-fdfe4c6cb404 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:15.985456 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 131/255] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:15.996576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 125/256] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:16.008571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8047e-e550-4660-a80c-38c2319ab721 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:16.008906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8047e-e550-4660-a80c-38c2319ab721 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:16.009139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8047e-e550-4660-a80c-38c2319ab721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:16.033641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c239cc3-2e61-4080-bdfd-5aef3ff1e0f5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:16.038436 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cae8047e-e550-4660-a80c-38c2319ab721 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:16.039644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cae8047e-e550-4660-a80c-38c2319ab721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:16.136921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 132/257] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:16.169090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8047e-e550-4660-a80c-38c2319ab721 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:16.222493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:16.223689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c239cc3-2e61-4080-bdfd-5aef3ff1e0f5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:16.280731 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca410169-f6be-4e52-8d8c-f10037c6c566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:16.282043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca410169-f6be-4e52-8d8c-f10037c6c566 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:16.283760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca410169-f6be-4e52-8d8c-f10037c6c566 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:16.284369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca410169-f6be-4e52-8d8c-f10037c6c566 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:16.305891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca410169-f6be-4e52-8d8c-f10037c6c566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ca410169-f6be-4e52-8d8c-f10037c6c566 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:16.320247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:16.321378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:16.322398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:16.327496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 126/258] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:16.342220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca410169-f6be-4e52-8d8c-f10037c6c566 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:16.349462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8047e-e550-4660-a80c-38c2319ab721 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:16.386127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbefa7-5c72-4c4a-a3e2-548fc94a6b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:16.387116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbefa7-5c72-4c4a-a3e2-548fc94a6b47 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:16.387399 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbefa7-5c72-4c4a-a3e2-548fc94a6b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:16.387703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbefa7-5c72-4c4a-a3e2-548fc94a6b47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:16.402122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbefa7-5c72-4c4a-a3e2-548fc94a6b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e7dbefa7-5c72-4c4a-a3e2-548fc94a6b47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:16.406401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:16.407526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:16.408407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 133/259] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:51:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:16.413664 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbefa7-5c72-4c4a-a3e2-548fc94a6b47 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:16.421688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:16.520111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca410169-f6be-4e52-8d8c-f10037c6c566 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:16.545734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:16.553613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbefa7-5c72-4c4a-a3e2-548fc94a6b47 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:16.591070 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff029b-3daf-4f88-a05a-43ec7c32452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:16.591758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff029b-3daf-4f88-a05a-43ec7c32452b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:16.592039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff029b-3daf-4f88-a05a-43ec7c32452b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:16.592311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff029b-3daf-4f88-a05a-43ec7c32452b None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:16.593073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff029b-3daf-4f88-a05a-43ec7c32452b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:16.593382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff029b-3daf-4f88-a05a-43ec7c32452b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:16.594811 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff029b-3daf-4f88-a05a-43ec7c32452b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:16.614605 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 127/260] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:16.618246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6b1cc-8b24-4654-9f6a-31c83971a839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:16.618936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6b1cc-8b24-4654-9f6a-31c83971a839 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:16.619217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6b1cc-8b24-4654-9f6a-31c83971a839 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:16.619498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6b1cc-8b24-4654-9f6a-31c83971a839 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:16.620405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6b1cc-8b24-4654-9f6a-31c83971a839 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:16.621004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6b1cc-8b24-4654-9f6a-31c83971a839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:16.622144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff029b-3daf-4f88-a05a-43ec7c32452b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:16.626301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6b1cc-8b24-4654-9f6a-31c83971a839 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:16.644610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 134/261] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:16.650686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6b1cc-8b24-4654-9f6a-31c83971a839 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:16.753188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff029b-3daf-4f88-a05a-43ec7c32452b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:16.762081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6b1cc-8b24-4654-9f6a-31c83971a839 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:16.765620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:16.799590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58ce5-c689-4cc6-9724-3731a1e52bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:16.800729 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58ce5-c689-4cc6-9724-3731a1e52bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:16.801248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58ce5-c689-4cc6-9724-3731a1e52bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:16.801670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58ce5-c689-4cc6-9724-3731a1e52bf9 None admin] PATH_INFO: `/v3/users/c90805ff76184accb2a79910a58a01d2` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:16.803128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d58ce5-c689-4cc6-9724-3731a1e52bf9 None admin] RBAC: Authorizing `identity:get_user(user_id=c90805ff76184accb2a79910a58a01d2)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:16.804719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d58ce5-c689-4cc6-9724-3731a1e52bf9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:16.806918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 135/262] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:16 2022] GET /identity/v3/users/c90805ff76184accb2a79910a58a01d2 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:16.810110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c64b59-7f20-4b21-b9c3-e9d05a2af8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:16.810110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c64b59-7f20-4b21-b9c3-e9d05a2af8de None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:16.810110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c64b59-7f20-4b21-b9c3-e9d05a2af8de None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:16.810110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c64b59-7f20-4b21-b9c3-e9d05a2af8de None admin] PATH_INFO: `/v3/users/c90805ff76184accb2a79910a58a01d2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:16.810451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c64b59-7f20-4b21-b9c3-e9d05a2af8de None admin] RBAC: Authorizing `identity:get_user(user_id=c90805ff76184accb2a79910a58a01d2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:16.811876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c64b59-7f20-4b21-b9c3-e9d05a2af8de None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:16.812988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58ce5-c689-4cc6-9724-3731a1e52bf9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:16.815770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 128/263] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:16 2022] GET /identity/v3/users/c90805ff76184accb2a79910a58a01d2 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:16.823871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c64b59-7f20-4b21-b9c3-e9d05a2af8de None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:16.866582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:17.018071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c64b59-7f20-4b21-b9c3-e9d05a2af8de None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:17.065779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd6b50-dfe2-4d1d-9479-13b35774487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:17.066516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd6b50-dfe2-4d1d-9479-13b35774487d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:17.066800 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd6b50-dfe2-4d1d-9479-13b35774487d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:17.067075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd6b50-dfe2-4d1d-9479-13b35774487d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:17.078802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58ce5-c689-4cc6-9724-3731a1e52bf9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:17.144790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b5a9f-2a5c-475a-bb5e-221eff85c19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:17.145503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b5a9f-2a5c-475a-bb5e-221eff85c19c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:17.145785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b5a9f-2a5c-475a-bb5e-221eff85c19c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:17.146121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b5a9f-2a5c-475a-bb5e-221eff85c19c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:17.158450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:17.206374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd6b50-dfe2-4d1d-9479-13b35774487d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:17.207918 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd6b50-dfe2-4d1d-9479-13b35774487d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:17.287005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b5a9f-2a5c-475a-bb5e-221eff85c19c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:17.288481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b5a9f-2a5c-475a-bb5e-221eff85c19c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:17.320830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:17.407608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 136/264] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 10:51:17.415103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b5a9f-2a5c-475a-bb5e-221eff85c19c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:17.441651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 129/265] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 10:51:17.446296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd6b50-dfe2-4d1d-9479-13b35774487d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:17.526471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:17.548942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd6b50-dfe2-4d1d-9479-13b35774487d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:17.583815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b5a9f-2a5c-475a-bb5e-221eff85c19c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:17.594632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46aa57-932d-4244-9b4b-05565cc4877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:17.595515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46aa57-932d-4244-9b4b-05565cc4877c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:17.595794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46aa57-932d-4244-9b4b-05565cc4877c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:17.596067 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46aa57-932d-4244-9b4b-05565cc4877c None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:17.597318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46aa57-932d-4244-9b4b-05565cc4877c None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:17.598917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af46aa57-932d-4244-9b4b-05565cc4877c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:17.605588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 130/266] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:17 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:17.609199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46aa57-932d-4244-9b4b-05565cc4877c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:17.640163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:17.662909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074f4cc-eebb-42e4-adc2-af96a37598f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:17.663802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074f4cc-eebb-42e4-adc2-af96a37598f8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:17.664078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074f4cc-eebb-42e4-adc2-af96a37598f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:17.664351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074f4cc-eebb-42e4-adc2-af96a37598f8 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:17.667478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074f4cc-eebb-42e4-adc2-af96a37598f8 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:17.669809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074f4cc-eebb-42e4-adc2-af96a37598f8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:17.678925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 137/267] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:17 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:17.685107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074f4cc-eebb-42e4-adc2-af96a37598f8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:17.750415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46aa57-932d-4244-9b4b-05565cc4877c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:17.785759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204ebb9-504e-45ea-9386-90cc71f4e4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:17.786501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204ebb9-504e-45ea-9386-90cc71f4e4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:17.786789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204ebb9-504e-45ea-9386-90cc71f4e4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:17.787061 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204ebb9-504e-45ea-9386-90cc71f4e4c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:17.787906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204ebb9-504e-45ea-9386-90cc71f4e4c6 None admin] RBAC: Adding query filter params (role.id=ffab93457c0f4f64a29764f17759396e, scope.project.id=205237c47ce44b8b839b43198fb4c1f1, user.id=c90805ff76184accb2a79910a58a01d2) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:17.788209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204ebb9-504e-45ea-9386-90cc71f4e4c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:17.789667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204ebb9-504e-45ea-9386-90cc71f4e4c6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:17.799680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 131/268] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:17 2022] GET /identity/v3/role_assignments?user.id=c90805ff76184accb2a79910a58a01d2&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=ffab93457c0f4f64a29764f17759396e => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:17.907788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074f4cc-eebb-42e4-adc2-af96a37598f8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:17.962254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93beaf87-8b5d-427f-aa92-f480d57f898a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:17.963384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beaf87-8b5d-427f-aa92-f480d57f898a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:17.963883 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beaf87-8b5d-427f-aa92-f480d57f898a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:17.964362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beaf87-8b5d-427f-aa92-f480d57f898a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:17.965708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beaf87-8b5d-427f-aa92-f480d57f898a None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=205237c47ce44b8b839b43198fb4c1f1, user.id=c90805ff76184accb2a79910a58a01d2) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:17.966022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beaf87-8b5d-427f-aa92-f480d57f898a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:17.967522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beaf87-8b5d-427f-aa92-f480d57f898a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:17.974309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 138/269] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:17 2022] GET /identity/v3/role_assignments?user.id=c90805ff76184accb2a79910a58a01d2&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:18.290992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a0cec-af8f-4619-84a5-f5374bdb8123 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.291499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a0cec-af8f-4619-84a5-f5374bdb8123 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.291574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a0cec-af8f-4619-84a5-f5374bdb8123 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.293370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 132/270] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:18.302117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5f77a-9172-4c13-9b7c-c2eda28dfad6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.302585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5f77a-9172-4c13-9b7c-c2eda28dfad6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.302775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5f77a-9172-4c13-9b7c-c2eda28dfad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.322002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bee5f77a-9172-4c13-9b7c-c2eda28dfad6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:18.323286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bee5f77a-9172-4c13-9b7c-c2eda28dfad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:18.386535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:18.410117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 139/271] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:18.434573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a0cec-af8f-4619-84a5-f5374bdb8123 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:18.450818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:18.522594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68439289-39be-41cf-9134-9b6a9a905f2a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.523247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68439289-39be-41cf-9134-9b6a9a905f2a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.523730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68439289-39be-41cf-9134-9b6a9a905f2a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.525747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 140/272] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:18.531681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f70c5c-4fff-45ac-976d-7d8cac57ce48 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.532279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f70c5c-4fff-45ac-976d-7d8cac57ce48 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.532767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f70c5c-4fff-45ac-976d-7d8cac57ce48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.553948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-52f70c5c-4fff-45ac-976d-7d8cac57ce48 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:18.555604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-52f70c5c-4fff-45ac-976d-7d8cac57ce48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:18.570942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a0cec-af8f-4619-84a5-f5374bdb8123 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:18.623413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95f491-b881-4e0e-8ee2-368d9748a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:18.624627 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95f491-b881-4e0e-8ee2-368d9748a08f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.625104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95f491-b881-4e0e-8ee2-368d9748a08f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.625572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95f491-b881-4e0e-8ee2-368d9748a08f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.637562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95f491-b881-4e0e-8ee2-368d9748a08f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:18.646109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 141/273] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ad95f491-b881-4e0e-8ee2-368d9748a08f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:18.647128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:18.647970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:18.648763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 133/274] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:18.655358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f70c5c-4fff-45ac-976d-7d8cac57ce48 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:18.667089 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95f491-b881-4e0e-8ee2-368d9748a08f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:18.744087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:18.760813 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f70c5c-4fff-45ac-976d-7d8cac57ce48 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:18.775945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:18.787553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95f491-b881-4e0e-8ee2-368d9748a08f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:18.801752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6f68d-6391-4845-a5d9-9569c4e751a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:18.802467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6f68d-6391-4845-a5d9-9569c4e751a9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.802811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6f68d-6391-4845-a5d9-9569c4e751a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.803025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6f68d-6391-4845-a5d9-9569c4e751a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.803856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6f68d-6391-4845-a5d9-9569c4e751a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:18.804176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6f68d-6391-4845-a5d9-9569c4e751a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:18.805530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6f68d-6391-4845-a5d9-9569c4e751a9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:18.816442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 142/275] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:18.816442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e43e1-3f15-4179-aa6e-973d38da3c6a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.816442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e43e1-3f15-4179-aa6e-973d38da3c6a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.816442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e43e1-3f15-4179-aa6e-973d38da3c6a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.816442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 143/276] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:18.816442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cde05-7e72-42b3-92a8-0c0fc73d17dc None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.816442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cde05-7e72-42b3-92a8-0c0fc73d17dc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.821429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cde05-7e72-42b3-92a8-0c0fc73d17dc None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.822514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 144/277] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:18.824089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cde05-7e72-42b3-92a8-0c0fc73d17dc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:18.828636 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19bc38-dfa2-4084-8303-62715e053095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:18.829446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19bc38-dfa2-4084-8303-62715e053095 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.829728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19bc38-dfa2-4084-8303-62715e053095 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.830067 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19bc38-dfa2-4084-8303-62715e053095 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.841341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19bc38-dfa2-4084-8303-62715e053095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-dc19bc38-dfa2-4084-8303-62715e053095 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:18.849093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:18.850063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 134/278] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:18.855999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50381540-66ca-4686-9dfa-d993983c8a89 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.856732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50381540-66ca-4686-9dfa-d993983c8a89 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.856732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50381540-66ca-4686-9dfa-d993983c8a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.869025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-50381540-66ca-4686-9dfa-d993983c8a89 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:18.870259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-50381540-66ca-4686-9dfa-d993983c8a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:18.924716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cde05-7e72-42b3-92a8-0c0fc73d17dc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:18.939321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 135/279] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:18.940595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0f750-ab16-4d01-be18-b7f407f9cebe None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.940875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0f750-ab16-4d01-be18-b7f407f9cebe None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.941146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0f750-ab16-4d01-be18-b7f407f9cebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.959783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b64bc-aeb2-4ee7-a7bc-3321c493fe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:18.960667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b64bc-aeb2-4ee7-a7bc-3321c493fe5a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:18.960950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b64bc-aeb2-4ee7-a7bc-3321c493fe5a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:18.961225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b64bc-aeb2-4ee7-a7bc-3321c493fe5a None admin] PATH_INFO: `/v3/users/5ce319702d7a45daa2a3aeb0f226479d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:18.970198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-aea0f750-ab16-4d01-be18-b7f407f9cebe None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:18.971376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-aea0f750-ab16-4d01-be18-b7f407f9cebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:18.975448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b64bc-aeb2-4ee7-a7bc-3321c493fe5a None admin] RBAC: Authorizing `identity:get_user(user_id=5ce319702d7a45daa2a3aeb0f226479d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:18.976983 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b64bc-aeb2-4ee7-a7bc-3321c493fe5a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:18.978711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 145/280] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:18 2022] GET /identity/v3/users/5ce319702d7a45daa2a3aeb0f226479d => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:18.979481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b64bc-aeb2-4ee7-a7bc-3321c493fe5a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.013962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:19.029987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:19.041194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 136/281] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:19.043051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0f750-ab16-4d01-be18-b7f407f9cebe None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.079016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b64bc-aeb2-4ee7-a7bc-3321c493fe5a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.112661 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b617f8-9ffe-477c-a91d-9e63176801d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.113363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b617f8-9ffe-477c-a91d-9e63176801d3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.113643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b617f8-9ffe-477c-a91d-9e63176801d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.113918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b617f8-9ffe-477c-a91d-9e63176801d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:19.114725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b617f8-9ffe-477c-a91d-9e63176801d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:19.115034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b617f8-9ffe-477c-a91d-9e63176801d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:19.116470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b617f8-9ffe-477c-a91d-9e63176801d3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:19.124974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 146/282] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:19.124974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b617f8-9ffe-477c-a91d-9e63176801d3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.145436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0f750-ab16-4d01-be18-b7f407f9cebe None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.178138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31428b92-269d-4331-90d7-f9c2737a5d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.178942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31428b92-269d-4331-90d7-f9c2737a5d06 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.179225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31428b92-269d-4331-90d7-f9c2737a5d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.179501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31428b92-269d-4331-90d7-f9c2737a5d06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:19.180655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31428b92-269d-4331-90d7-f9c2737a5d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:19.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31428b92-269d-4331-90d7-f9c2737a5d06 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:19.183854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 137/283] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:19.184774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31428b92-269d-4331-90d7-f9c2737a5d06 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.217870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:19.224235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b617f8-9ffe-477c-a91d-9e63176801d3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.246575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:19.259439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53406c-7f2c-44aa-bbc5-344e4504ed62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.260103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53406c-7f2c-44aa-bbc5-344e4504ed62 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.260420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53406c-7f2c-44aa-bbc5-344e4504ed62 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.260698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53406c-7f2c-44aa-bbc5-344e4504ed62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:19.261737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53406c-7f2c-44aa-bbc5-344e4504ed62 None admin] RBAC: Adding query filter params (role.id=3e46d43e30ef494087da2c44f7ca702f, user.id=5ce319702d7a45daa2a3aeb0f226479d, scope.system=all) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:19.261928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53406c-7f2c-44aa-bbc5-344e4504ed62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:19.263564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53406c-7f2c-44aa-bbc5-344e4504ed62 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:19.268681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 147/284] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jun 20 10:51:19 2022] GET /identity/v3/role_assignments?user.id=5ce319702d7a45daa2a3aeb0f226479d&scope.system=all&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:19.269461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53406c-7f2c-44aa-bbc5-344e4504ed62 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.294345 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31428b92-269d-4331-90d7-f9c2737a5d06 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.337306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c750563-d480-46ea-94c2-e808ae9743cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.338427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c750563-d480-46ea-94c2-e808ae9743cf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.338713 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c750563-d480-46ea-94c2-e808ae9743cf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.338989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c750563-d480-46ea-94c2-e808ae9743cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:19.351449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c750563-d480-46ea-94c2-e808ae9743cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0c750563-d480-46ea-94c2-e808ae9743cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:19.360094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:19.361110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:19.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 138/285] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:19.362589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c750563-d480-46ea-94c2-e808ae9743cf None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.384147 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53406c-7f2c-44aa-bbc5-344e4504ed62 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.423073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69eb55-241d-44ec-bad2-9b783d3e441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.423971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69eb55-241d-44ec-bad2-9b783d3e441d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.424258 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69eb55-241d-44ec-bad2-9b783d3e441d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.424533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69eb55-241d-44ec-bad2-9b783d3e441d None admin] PATH_INFO: `/v3/users/dce50a7df665449baa3e629e32f6ef75` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:19.425759 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69eb55-241d-44ec-bad2-9b783d3e441d None admin] RBAC: Authorizing `identity:get_user(user_id=dce50a7df665449baa3e629e32f6ef75)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:19.427634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69eb55-241d-44ec-bad2-9b783d3e441d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:19.428913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 148/286] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:19 2022] GET /identity/v3/users/dce50a7df665449baa3e629e32f6ef75 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:19.430567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69eb55-241d-44ec-bad2-9b783d3e441d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.462157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:19.486905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c750563-d480-46ea-94c2-e808ae9743cf None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.507179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:19.525668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e7451-3fa6-4f6e-88c8-3832ae6c6d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.526457 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e7451-3fa6-4f6e-88c8-3832ae6c6d50 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.526743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e7451-3fa6-4f6e-88c8-3832ae6c6d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.527023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e7451-3fa6-4f6e-88c8-3832ae6c6d50 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:19.527779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e7451-3fa6-4f6e-88c8-3832ae6c6d50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:19.529499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e7451-3fa6-4f6e-88c8-3832ae6c6d50 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:19.534476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69eb55-241d-44ec-bad2-9b783d3e441d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.553331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 139/287] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:51:19 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:19.553980 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e7451-3fa6-4f6e-88c8-3832ae6c6d50 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.584222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a899d4-a11f-4e2e-8a2a-155e3d3c7476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.585352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a899d4-a11f-4e2e-8a2a-155e3d3c7476 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.585925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a899d4-a11f-4e2e-8a2a-155e3d3c7476 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.586572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a899d4-a11f-4e2e-8a2a-155e3d3c7476 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:19.587618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a899d4-a11f-4e2e-8a2a-155e3d3c7476 None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:19.588136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a899d4-a11f-4e2e-8a2a-155e3d3c7476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:19.590047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a899d4-a11f-4e2e-8a2a-155e3d3c7476 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:19.600397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 149/288] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:19.606345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a899d4-a11f-4e2e-8a2a-155e3d3c7476 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.678448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e7451-3fa6-4f6e-88c8-3832ae6c6d50 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.712703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:19.720063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331409e9-357c-40c4-a3db-d8a39d18cc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.721221 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331409e9-357c-40c4-a3db-d8a39d18cc35 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.721502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331409e9-357c-40c4-a3db-d8a39d18cc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.721779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331409e9-357c-40c4-a3db-d8a39d18cc35 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:19.723188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331409e9-357c-40c4-a3db-d8a39d18cc35 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:19.725261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331409e9-357c-40c4-a3db-d8a39d18cc35 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:19.727402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a899d4-a11f-4e2e-8a2a-155e3d3c7476 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.727520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 140/289] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:19 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:19.735063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331409e9-357c-40c4-a3db-d8a39d18cc35 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.772571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d514a6-cdef-4968-af99-69eca23a72e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d514a6-cdef-4968-af99-69eca23a72e7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.774058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d514a6-cdef-4968-af99-69eca23a72e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.774364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d514a6-cdef-4968-af99-69eca23a72e7 None admin] PATH_INFO: `/v3/users/6e32fe77ebc04e8ca85f4c322eae083e` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:19.775604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d514a6-cdef-4968-af99-69eca23a72e7 None admin] RBAC: Authorizing `identity:get_user(user_id=6e32fe77ebc04e8ca85f4c322eae083e)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:19.777058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d514a6-cdef-4968-af99-69eca23a72e7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:19.782122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 150/290] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:19 2022] GET /identity/v3/users/6e32fe77ebc04e8ca85f4c322eae083e => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:19.785338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d514a6-cdef-4968-af99-69eca23a72e7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:19.801164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:19.885387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331409e9-357c-40c4-a3db-d8a39d18cc35 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.926017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d514a6-cdef-4968-af99-69eca23a72e7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:19.941788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f226a-bbb3-4312-b1d3-357fb6a6b3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.943335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f226a-bbb3-4312-b1d3-357fb6a6b3a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.944298 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f226a-bbb3-4312-b1d3-357fb6a6b3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.944890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f226a-bbb3-4312-b1d3-357fb6a6b3a3 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1/users/dce50a7df665449baa3e629e32f6ef75/roles/e4214ef8456541528194f90e229b356a` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:19.947895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f226a-bbb3-4312-b1d3-357fb6a6b3a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=205237c47ce44b8b839b43198fb4c1f1, user_id=dce50a7df665449baa3e629e32f6ef75, role_id=e4214ef8456541528194f90e229b356a)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:19.949667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f226a-bbb3-4312-b1d3-357fb6a6b3a3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:19.968608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 141/291] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:19 2022] PUT /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1/users/dce50a7df665449baa3e629e32f6ef75/roles/e4214ef8456541528194f90e229b356a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:19.971153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed1672-6fa7-4fc7-92ad-9635959b16b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:19.971893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed1672-6fa7-4fc7-92ad-9635959b16b8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:19.972289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed1672-6fa7-4fc7-92ad-9635959b16b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:19.972635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed1672-6fa7-4fc7-92ad-9635959b16b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:20.007614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:20.056349 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed1672-6fa7-4fc7-92ad-9635959b16b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:20.057935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed1672-6fa7-4fc7-92ad-9635959b16b8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:20.171681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 151/292] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 10:51:20.177126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f226a-bbb3-4312-b1d3-357fb6a6b3a3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:20.302456 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f226a-bbb3-4312-b1d3-357fb6a6b3a3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:20.304154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:20.339709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b5604-10f2-484e-9f2d-157cae343215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:20.340594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b5604-10f2-484e-9f2d-157cae343215 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:20.340880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b5604-10f2-484e-9f2d-157cae343215 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:20.341160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b5604-10f2-484e-9f2d-157cae343215 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:20.342408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b5604-10f2-484e-9f2d-157cae343215 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:20.343892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b5604-10f2-484e-9f2d-157cae343215 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:20.345946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 142/293] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:20 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:20.351484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed1672-6fa7-4fc7-92ad-9635959b16b8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:20.431886 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:20.485149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed1672-6fa7-4fc7-92ad-9635959b16b8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:20.535884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23f244-842a-4cff-b50b-fd0f5969ad53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:20.536566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23f244-842a-4cff-b50b-fd0f5969ad53 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:20.536848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23f244-842a-4cff-b50b-fd0f5969ad53 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:20.537122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23f244-842a-4cff-b50b-fd0f5969ad53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:20.537970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23f244-842a-4cff-b50b-fd0f5969ad53 None admin] RBAC: Adding query filter params (role.id=3e46d43e30ef494087da2c44f7ca702f, scope.project.id=205237c47ce44b8b839b43198fb4c1f1, user.id=6e32fe77ebc04e8ca85f4c322eae083e) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:20.538317 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23f244-842a-4cff-b50b-fd0f5969ad53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:20.539780 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23f244-842a-4cff-b50b-fd0f5969ad53 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:20.546126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 152/294] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:20 2022] GET /identity/v3/role_assignments?user.id=6e32fe77ebc04e8ca85f4c322eae083e&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:21.426615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7a42-b221-4541-bd31-ca867af85470 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:21.427161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7a42-b221-4541-bd31-ca867af85470 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:21.427161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7a42-b221-4541-bd31-ca867af85470 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:21.428855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 143/295] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:21.434039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4fd66-d81a-4dfb-9c65-e95510bb5151 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:21.434440 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4fd66-d81a-4dfb-9c65-e95510bb5151 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:21.434635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4fd66-d81a-4dfb-9c65-e95510bb5151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:21.451520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-21f4fd66-d81a-4dfb-9c65-e95510bb5151 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:21.452729 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-21f4fd66-d81a-4dfb-9c65-e95510bb5151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:21.523008 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 153/296] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:21.546302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e7a42-b221-4541-bd31-ca867af85470 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:21.658202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e7a42-b221-4541-bd31-ca867af85470 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:21.693759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd301f-5d56-4cd6-9c53-e357d9e8d983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:21.694527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd301f-5d56-4cd6-9c53-e357d9e8d983 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:21.694869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd301f-5d56-4cd6-9c53-e357d9e8d983 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:21.695149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd301f-5d56-4cd6-9c53-e357d9e8d983 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:21.706278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd301f-5d56-4cd6-9c53-e357d9e8d983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-81cd301f-5d56-4cd6-9c53-e357d9e8d983 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:21.714543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:21.715751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:21.717082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 144/297] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:21.721897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4fd66-d81a-4dfb-9c65-e95510bb5151 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:21.776058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:21.825423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4fd66-d81a-4dfb-9c65-e95510bb5151 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:21.860325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24befed6-a754-4750-b42a-a7740a075d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:21.861006 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24befed6-a754-4750-b42a-a7740a075d57 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:21.861291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24befed6-a754-4750-b42a-a7740a075d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:21.861570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24befed6-a754-4750-b42a-a7740a075d57 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:21.862412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24befed6-a754-4750-b42a-a7740a075d57 None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:21.862724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24befed6-a754-4750-b42a-a7740a075d57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:21.864156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24befed6-a754-4750-b42a-a7740a075d57 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:21.872701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 154/298] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:21.878301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd301f-5d56-4cd6-9c53-e357d9e8d983 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:21.912966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:21.984617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd301f-5d56-4cd6-9c53-e357d9e8d983 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:22.018933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251248e3-adf5-43f3-83de-6e256129c87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:22.019842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251248e3-adf5-43f3-83de-6e256129c87a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.020127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251248e3-adf5-43f3-83de-6e256129c87a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.020403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251248e3-adf5-43f3-83de-6e256129c87a None admin] PATH_INFO: `/v3/users/5ce319702d7a45daa2a3aeb0f226479d` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.021612 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251248e3-adf5-43f3-83de-6e256129c87a None admin] RBAC: Authorizing `identity:get_user(user_id=5ce319702d7a45daa2a3aeb0f226479d)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:22.023164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251248e3-adf5-43f3-83de-6e256129c87a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:22.024833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 145/299] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:21 2022] GET /identity/v3/users/5ce319702d7a45daa2a3aeb0f226479d => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:22.030641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24befed6-a754-4750-b42a-a7740a075d57 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:22.130343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4bf5e-850c-49ce-871a-c4e328b1a69c None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.130687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4bf5e-850c-49ce-871a-c4e328b1a69c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.130975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4bf5e-850c-49ce-871a-c4e328b1a69c None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.132644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 146/300] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:22.134831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24befed6-a754-4750-b42a-a7740a075d57 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:22.137759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaaad82-9608-4ac7-8ace-f0582df12143 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.138047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaaad82-9608-4ac7-8ace-f0582df12143 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.138362 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaaad82-9608-4ac7-8ace-f0582df12143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.155522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-acaaad82-9608-4ac7-8ace-f0582df12143 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:22.156739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-acaaad82-9608-4ac7-8ace-f0582df12143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:22.158555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:22.170319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d8e26-51b2-45c4-852c-dca18fa5ccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:22.171206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d8e26-51b2-45c4-852c-dca18fa5ccec None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.171486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d8e26-51b2-45c4-852c-dca18fa5ccec None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.171763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d8e26-51b2-45c4-852c-dca18fa5ccec None admin] PATH_INFO: `/v3/system/users/5ce319702d7a45daa2a3aeb0f226479d/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.173247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d8e26-51b2-45c4-852c-dca18fa5ccec None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5ce319702d7a45daa2a3aeb0f226479d, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:22.174672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d8e26-51b2-45c4-852c-dca18fa5ccec None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:22.184488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 155/301] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Jun 20 10:51:22 2022] PUT /identity/v3/system/users/5ce319702d7a45daa2a3aeb0f226479d/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:22.186040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d138-e0f9-44df-b9d2-2cb7b326b918 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.186355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d138-e0f9-44df-b9d2-2cb7b326b918 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.186628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d138-e0f9-44df-b9d2-2cb7b326b918 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.187922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 156/302] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:22.194721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e684947-6776-4acb-b40d-0ccc6db2d5e5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.195000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e684947-6776-4acb-b40d-0ccc6db2d5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.195276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e684947-6776-4acb-b40d-0ccc6db2d5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.219399 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3e684947-6776-4acb-b40d-0ccc6db2d5e5 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:22.221211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3e684947-6776-4acb-b40d-0ccc6db2d5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:22.255223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 147/303] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:22.276340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaaad82-9608-4ac7-8ace-f0582df12143 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:22.306310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 157/304] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:22.308059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41666d-287d-4ab0-af39-27c52c2e6f33 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.308609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41666d-287d-4ab0-af39-27c52c2e6f33 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.309086 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41666d-287d-4ab0-af39-27c52c2e6f33 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.310911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 158/305] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:22.316600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111b31c-7a42-4d5b-bb63-cd1354818d12 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.317227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111b31c-7a42-4d5b-bb63-cd1354818d12 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.317739 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111b31c-7a42-4d5b-bb63-cd1354818d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.319314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:22.338716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b111b31c-7a42-4d5b-bb63-cd1354818d12 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:22.340124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b111b31c-7a42-4d5b-bb63-cd1354818d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:22.409929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaaad82-9608-4ac7-8ace-f0582df12143 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:22.410865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 159/306] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:22.411722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111b31c-7a42-4d5b-bb63-cd1354818d12 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:22.453841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445639f-a878-4b61-ae6b-e61aae28b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:22.454740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445639f-a878-4b61-ae6b-e61aae28b25c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.455026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445639f-a878-4b61-ae6b-e61aae28b25c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.455306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445639f-a878-4b61-ae6b-e61aae28b25c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.456482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445639f-a878-4b61-ae6b-e61aae28b25c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:22.457927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445639f-a878-4b61-ae6b-e61aae28b25c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:22.459179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:22.464707 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 148/307] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:22.465706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445639f-a878-4b61-ae6b-e61aae28b25c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:22.526428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111b31c-7a42-4d5b-bb63-cd1354818d12 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:22.563084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:22.564021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46887b40-0e33-4d0e-b8a3-1c59c343b898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:22.564889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46887b40-0e33-4d0e-b8a3-1c59c343b898 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.565170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46887b40-0e33-4d0e-b8a3-1c59c343b898 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.565431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46887b40-0e33-4d0e-b8a3-1c59c343b898 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.569930 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445639f-a878-4b61-ae6b-e61aae28b25c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:22.576080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46887b40-0e33-4d0e-b8a3-1c59c343b898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-46887b40-0e33-4d0e-b8a3-1c59c343b898 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:22.584293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:22.585384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 160/308] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:22.586781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46887b40-0e33-4d0e-b8a3-1c59c343b898 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:22.614473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbd6e2-e164-4b91-8fb6-a2a27a517549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:22.615282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbd6e2-e164-4b91-8fb6-a2a27a517549 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.615561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbd6e2-e164-4b91-8fb6-a2a27a517549 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.615832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbd6e2-e164-4b91-8fb6-a2a27a517549 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.627241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdbd6e2-e164-4b91-8fb6-a2a27a517549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-dcdbd6e2-e164-4b91-8fb6-a2a27a517549 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:22.634928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:22.635815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:22.637019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 149/309] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:22.637701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbd6e2-e164-4b91-8fb6-a2a27a517549 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:22.697568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:22.702726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46887b40-0e33-4d0e-b8a3-1c59c343b898 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:22.741359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669d191-eace-4ed6-87db-f5054ff08a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:22.742025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669d191-eace-4ed6-87db-f5054ff08a9f None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.742310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669d191-eace-4ed6-87db-f5054ff08a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.742606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669d191-eace-4ed6-87db-f5054ff08a9f None admin] PATH_INFO: `/v3/groups` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.743292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669d191-eace-4ed6-87db-f5054ff08a9f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:22.744742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669d191-eace-4ed6-87db-f5054ff08a9f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:22.747031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbd6e2-e164-4b91-8fb6-a2a27a517549 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:22.762596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 161/310] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Jun 20 10:51:22 2022] POST /identity/v3/groups => generated 229 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:22.762596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669d191-eace-4ed6-87db-f5054ff08a9f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:22.802618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a5c16-8c73-45f5-ab80-6d6ac48a5f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:22.803377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5c16-8c73-45f5-ab80-6d6ac48a5f5f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.803767 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5c16-8c73-45f5-ab80-6d6ac48a5f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.804052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5c16-8c73-45f5-ab80-6d6ac48a5f5f None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.804826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a5c16-8c73-45f5-ab80-6d6ac48a5f5f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:22.805259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a5c16-8c73-45f5-ab80-6d6ac48a5f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:22.806921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a5c16-8c73-45f5-ab80-6d6ac48a5f5f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:22.821217 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 150/311] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:22.822920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a5c16-8c73-45f5-ab80-6d6ac48a5f5f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:22.848147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:22.898760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669d191-eace-4ed6-87db-f5054ff08a9f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:22.947626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417f4f2-35fc-47d5-9201-9d512f786a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:22.948310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417f4f2-35fc-47d5-9201-9d512f786a3f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:22.948597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417f4f2-35fc-47d5-9201-9d512f786a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:22.948878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417f4f2-35fc-47d5-9201-9d512f786a3f None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:22.949781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417f4f2-35fc-47d5-9201-9d512f786a3f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:22.950205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417f4f2-35fc-47d5-9201-9d512f786a3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:22.951603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417f4f2-35fc-47d5-9201-9d512f786a3f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:22.962082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 162/312] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:22.967653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417f4f2-35fc-47d5-9201-9d512f786a3f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:22.971596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a5c16-8c73-45f5-ab80-6d6ac48a5f5f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:22.995278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:23.023490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65810c7a-2faf-4c98-851f-90f1d2d44071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:23.024395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65810c7a-2faf-4c98-851f-90f1d2d44071 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:23.024685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65810c7a-2faf-4c98-851f-90f1d2d44071 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:23.024963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65810c7a-2faf-4c98-851f-90f1d2d44071 None admin] PATH_INFO: `/v3/users/b467f816224740e5a2e9dc6a8736dd24` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:23.039461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65810c7a-2faf-4c98-851f-90f1d2d44071 None admin] RBAC: Authorizing `identity:get_user(user_id=b467f816224740e5a2e9dc6a8736dd24)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:23.040951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65810c7a-2faf-4c98-851f-90f1d2d44071 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:23.043253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 151/313] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:22 2022] GET /identity/v3/users/b467f816224740e5a2e9dc6a8736dd24 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:23.048524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65810c7a-2faf-4c98-851f-90f1d2d44071 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:23.097411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417f4f2-35fc-47d5-9201-9d512f786a3f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:23.141277 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf6dd0-9a55-4285-9bde-941833cea048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:23.143058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf6dd0-9a55-4285-9bde-941833cea048 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:23.143514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf6dd0-9a55-4285-9bde-941833cea048 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:23.144104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf6dd0-9a55-4285-9bde-941833cea048 None admin] PATH_INFO: `/v3/users/dce50a7df665449baa3e629e32f6ef75` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:23.146024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf6dd0-9a55-4285-9bde-941833cea048 None admin] RBAC: Authorizing `identity:get_user(user_id=dce50a7df665449baa3e629e32f6ef75)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:23.147397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65810c7a-2faf-4c98-851f-90f1d2d44071 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:23.147924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf6dd0-9a55-4285-9bde-941833cea048 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:23.150380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:23.151115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 163/314] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:22 2022] GET /identity/v3/users/dce50a7df665449baa3e629e32f6ef75 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:23.157348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf6dd0-9a55-4285-9bde-941833cea048 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:23.194024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1cf80-34fc-47f4-b45f-7f6dae3cd86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:23.194727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1cf80-34fc-47f4-b45f-7f6dae3cd86a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:23.195016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1cf80-34fc-47f4-b45f-7f6dae3cd86a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:23.196145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1cf80-34fc-47f4-b45f-7f6dae3cd86a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:23.197024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1cf80-34fc-47f4-b45f-7f6dae3cd86a None admin] RBAC: Adding query filter params (role.id=e4214ef8456541528194f90e229b356a, user.id=b467f816224740e5a2e9dc6a8736dd24, scope.system=all) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:23.197343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1cf80-34fc-47f4-b45f-7f6dae3cd86a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:23.198842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1cf80-34fc-47f4-b45f-7f6dae3cd86a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:23.206236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 152/315] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jun 20 10:51:23 2022] GET /identity/v3/role_assignments?user.id=b467f816224740e5a2e9dc6a8736dd24&scope.system=all&role.id=e4214ef8456541528194f90e229b356a => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:23.268696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:23.284227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf6dd0-9a55-4285-9bde-941833cea048 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:23.323108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6368cf-fbdc-47f0-9b9c-132f4b0fa834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:23.323793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6368cf-fbdc-47f0-9b9c-132f4b0fa834 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:23.324178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6368cf-fbdc-47f0-9b9c-132f4b0fa834 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:23.324474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6368cf-fbdc-47f0-9b9c-132f4b0fa834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:23.401032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6368cf-fbdc-47f0-9b9c-132f4b0fa834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:23.403032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6368cf-fbdc-47f0-9b9c-132f4b0fa834 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:23.492044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:23.505764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 164/316] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:23.511615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1cf80-34fc-47f4-b45f-7f6dae3cd86a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:23.616201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1cf80-34fc-47f4-b45f-7f6dae3cd86a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:23.651424 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447f636-08b8-496f-ae5e-6a1383d9ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:23.652696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447f636-08b8-496f-ae5e-6a1383d9ce5f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:23.653188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447f636-08b8-496f-ae5e-6a1383d9ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:23.653652 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447f636-08b8-496f-ae5e-6a1383d9ce5f None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:23.655091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447f636-08b8-496f-ae5e-6a1383d9ce5f None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:23.656737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447f636-08b8-496f-ae5e-6a1383d9ce5f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:23.658974 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 153/317] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:23 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:23.664928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6368cf-fbdc-47f0-9b9c-132f4b0fa834 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:23.766340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6368cf-fbdc-47f0-9b9c-132f4b0fa834 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:23.800805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a461038-e5ad-4168-9986-97a11d7cefbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:23.801817 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a461038-e5ad-4168-9986-97a11d7cefbc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:23.802338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a461038-e5ad-4168-9986-97a11d7cefbc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:23.802829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a461038-e5ad-4168-9986-97a11d7cefbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:23.803874 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a461038-e5ad-4168-9986-97a11d7cefbc None admin] RBAC: Adding query filter params (role.id=e4214ef8456541528194f90e229b356a, scope.project.id=205237c47ce44b8b839b43198fb4c1f1, user.id=dce50a7df665449baa3e629e32f6ef75) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:23.804381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a461038-e5ad-4168-9986-97a11d7cefbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:23.806024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a461038-e5ad-4168-9986-97a11d7cefbc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:23.812596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 165/318] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:23 2022] GET /identity/v3/role_assignments?user.id=dce50a7df665449baa3e629e32f6ef75&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=e4214ef8456541528194f90e229b356a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:24.283991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e632db5-9025-47fe-a93b-078e7b428735 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:24.284239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e632db5-9025-47fe-a93b-078e7b428735 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:24.284510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e632db5-9025-47fe-a93b-078e7b428735 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:24.286128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 154/319] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:24.291247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cad19f-fb1e-4861-b0f5-06058e56bbf3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:24.291507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cad19f-fb1e-4861-b0f5-06058e56bbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:24.291825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cad19f-fb1e-4861-b0f5-06058e56bbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:24.309864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-28cad19f-fb1e-4861-b0f5-06058e56bbf3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:24.311120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-28cad19f-fb1e-4861-b0f5-06058e56bbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:24.363325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:24.385599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 166/320] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:24.408815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e632db5-9025-47fe-a93b-078e7b428735 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:24.421637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:24.517729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e632db5-9025-47fe-a93b-078e7b428735 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:24.552246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41613e-38dc-4bc6-8edb-571b732b4ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:24.552973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41613e-38dc-4bc6-8edb-571b732b4ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:24.553246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41613e-38dc-4bc6-8edb-571b732b4ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:24.553515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41613e-38dc-4bc6-8edb-571b732b4ee2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:24.563764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41613e-38dc-4bc6-8edb-571b732b4ee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-db41613e-38dc-4bc6-8edb-571b732b4ee2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:24.571370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:24.572234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:24.573018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 155/321] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:24.579915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cad19f-fb1e-4861-b0f5-06058e56bbf3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:24.684524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cad19f-fb1e-4861-b0f5-06058e56bbf3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:24.718856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de84a9f-ad11-4449-867a-410264acb51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:24.719529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de84a9f-ad11-4449-867a-410264acb51f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:24.719802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de84a9f-ad11-4449-867a-410264acb51f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:24.720074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de84a9f-ad11-4449-867a-410264acb51f None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:24.720816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de84a9f-ad11-4449-867a-410264acb51f None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:24.721119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de84a9f-ad11-4449-867a-410264acb51f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:24.722520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de84a9f-ad11-4449-867a-410264acb51f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:24.731099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 167/322] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:24.736492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41613e-38dc-4bc6-8edb-571b732b4ee2 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:24.802966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:24.838841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41613e-38dc-4bc6-8edb-571b732b4ee2 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:24.872394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d77d1-f2a9-440b-ac78-c08803e65c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:24.873342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d77d1-f2a9-440b-ac78-c08803e65c45 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:24.873622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d77d1-f2a9-440b-ac78-c08803e65c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:24.873922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d77d1-f2a9-440b-ac78-c08803e65c45 None admin] PATH_INFO: `/v3/users/5ce319702d7a45daa2a3aeb0f226479d` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:24.875163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d77d1-f2a9-440b-ac78-c08803e65c45 None admin] RBAC: Authorizing `identity:get_user(user_id=5ce319702d7a45daa2a3aeb0f226479d)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:24.876630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d77d1-f2a9-440b-ac78-c08803e65c45 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:24.878458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 156/323] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:24 2022] GET /identity/v3/users/5ce319702d7a45daa2a3aeb0f226479d => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:24.884085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de84a9f-ad11-4449-867a-410264acb51f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:24.938477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:24.940616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ceeae-28d7-4a16-b736-889c5b33d2e8 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:24.940905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ceeae-28d7-4a16-b736-889c5b33d2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:24.941310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ceeae-28d7-4a16-b736-889c5b33d2e8 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:24.942819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 157/324] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:24.947860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1112e67-c9a9-45c0-a7ca-e6987b4022ce None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:24.948149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1112e67-c9a9-45c0-a7ca-e6987b4022ce None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:24.948427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1112e67-c9a9-45c0-a7ca-e6987b4022ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:24.966802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f1112e67-c9a9-45c0-a7ca-e6987b4022ce None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:24.968091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f1112e67-c9a9-45c0-a7ca-e6987b4022ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:24.998875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de84a9f-ad11-4449-867a-410264acb51f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:25.034269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217a0ea-d5d8-4b73-8497-94f673e43ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:25.034937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217a0ea-d5d8-4b73-8497-94f673e43ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:25.035224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217a0ea-d5d8-4b73-8497-94f673e43ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:25.035498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217a0ea-d5d8-4b73-8497-94f673e43ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:25.036334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217a0ea-d5d8-4b73-8497-94f673e43ae2 None admin] RBAC: Adding query filter params (role.id=3e46d43e30ef494087da2c44f7ca702f, user.id=5ce319702d7a45daa2a3aeb0f226479d, scope.system=all) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:25.036643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217a0ea-d5d8-4b73-8497-94f673e43ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:25.038121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217a0ea-d5d8-4b73-8497-94f673e43ae2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:25.039457 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 158/325] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:25.054191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 168/326] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jun 20 10:51:24 2022] GET /identity/v3/role_assignments?user.id=5ce319702d7a45daa2a3aeb0f226479d&scope.system=all&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 510 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:25.061917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1112e67-c9a9-45c0-a7ca-e6987b4022ce None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:25.134453 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:25.174448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1112e67-c9a9-45c0-a7ca-e6987b4022ce None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:25.212624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75476af7-24d0-4255-86cf-3fe25f81f34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:25.213437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75476af7-24d0-4255-86cf-3fe25f81f34a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:25.213718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75476af7-24d0-4255-86cf-3fe25f81f34a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:25.214012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75476af7-24d0-4255-86cf-3fe25f81f34a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:25.215586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75476af7-24d0-4255-86cf-3fe25f81f34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:25.217051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75476af7-24d0-4255-86cf-3fe25f81f34a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:25.219099 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 159/327] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:25.224292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217a0ea-d5d8-4b73-8497-94f673e43ae2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:25.289034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f50006-84e8-4b99-83fb-c268c776d333 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:25.289321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f50006-84e8-4b99-83fb-c268c776d333 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:25.289604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f50006-84e8-4b99-83fb-c268c776d333 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:25.291246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 160/328] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:51:25.296131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedd2e1-8764-44fc-8ad6-131801a1f526 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:25.296412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedd2e1-8764-44fc-8ad6-131801a1f526 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:25.296688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedd2e1-8764-44fc-8ad6-131801a1f526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:25.313197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9aedd2e1-8764-44fc-8ad6-131801a1f526 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:25.314368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9aedd2e1-8764-44fc-8ad6-131801a1f526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:25.331764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217a0ea-d5d8-4b73-8497-94f673e43ae2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:25.333665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:25.366312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66cfe6-32a1-4181-b9d0-7d7d140c5b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:25.366991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66cfe6-32a1-4181-b9d0-7d7d140c5b92 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:25.367271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66cfe6-32a1-4181-b9d0-7d7d140c5b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:25.367545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66cfe6-32a1-4181-b9d0-7d7d140c5b92 None admin] PATH_INFO: `/v3/groups` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:25.368239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66cfe6-32a1-4181-b9d0-7d7d140c5b92 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:25.369686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66cfe6-32a1-4181-b9d0-7d7d140c5b92 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:25.380976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 161/329] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:25.402134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 169/330] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Jun 20 10:51:25 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:25.404244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedd2e1-8764-44fc-8ad6-131801a1f526 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:25.479004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:25.519725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedd2e1-8764-44fc-8ad6-131801a1f526 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:25.557495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090359e-6a87-4f62-8633-684e5514396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:25.557908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090359e-6a87-4f62-8633-684e5514396e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:25.558245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090359e-6a87-4f62-8633-684e5514396e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:25.558701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090359e-6a87-4f62-8633-684e5514396e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:25.569278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090359e-6a87-4f62-8633-684e5514396e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-a090359e-6a87-4f62-8633-684e5514396e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:25.577102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:25.577979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:25.578937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 162/331] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:25.584945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66cfe6-32a1-4181-b9d0-7d7d140c5b92 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:25.694163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66cfe6-32a1-4181-b9d0-7d7d140c5b92 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:25.730162 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fd3f2-632c-44d2-acbc-c560f8b4b9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:25.730909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fd3f2-632c-44d2-acbc-c560f8b4b9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:25.731182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fd3f2-632c-44d2-acbc-c560f8b4b9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:25.731454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fd3f2-632c-44d2-acbc-c560f8b4b9e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:25.732206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fd3f2-632c-44d2-acbc-c560f8b4b9e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:25.732520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fd3f2-632c-44d2-acbc-c560f8b4b9e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:25.733915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fd3f2-632c-44d2-acbc-c560f8b4b9e3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:25.743254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 170/332] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:25.749201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090359e-6a87-4f62-8633-684e5514396e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:25.857633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090359e-6a87-4f62-8633-684e5514396e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:25.882012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:25.895579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15795-411d-45d3-9b3a-6d078f14b158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:25.896476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15795-411d-45d3-9b3a-6d078f14b158 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:25.896758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15795-411d-45d3-9b3a-6d078f14b158 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:25.897032 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15795-411d-45d3-9b3a-6d078f14b158 None admin] PATH_INFO: `/v3/users/b467f816224740e5a2e9dc6a8736dd24` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:25.898335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15795-411d-45d3-9b3a-6d078f14b158 None admin] RBAC: Authorizing `identity:get_user(user_id=b467f816224740e5a2e9dc6a8736dd24)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:25.899816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15795-411d-45d3-9b3a-6d078f14b158 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:25.901814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 163/333] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:25 2022] GET /identity/v3/users/b467f816224740e5a2e9dc6a8736dd24 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:25.910604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fd3f2-632c-44d2-acbc-c560f8b4b9e3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:25.941289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:26.028660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fd3f2-632c-44d2-acbc-c560f8b4b9e3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:26.069312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebbbb6-25fe-40f0-a670-5f059e43b836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:26.070230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebbbb6-25fe-40f0-a670-5f059e43b836 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:26.070514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebbbb6-25fe-40f0-a670-5f059e43b836 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:26.070784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebbbb6-25fe-40f0-a670-5f059e43b836 None admin] PATH_INFO: `/v3/system/users/b467f816224740e5a2e9dc6a8736dd24/roles/e4214ef8456541528194f90e229b356a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:26.072281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebbbb6-25fe-40f0-a670-5f059e43b836 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b467f816224740e5a2e9dc6a8736dd24, role_id=e4214ef8456541528194f90e229b356a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:26.073682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebbbb6-25fe-40f0-a670-5f059e43b836 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:26.082965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 171/334] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Jun 20 10:51:25 2022] PUT /identity/v3/system/users/b467f816224740e5a2e9dc6a8736dd24/roles/e4214ef8456541528194f90e229b356a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:27.715922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661117e-a870-4581-89ee-481aff132573 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:27.716444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661117e-a870-4581-89ee-481aff132573 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:27.716551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661117e-a870-4581-89ee-481aff132573 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:27.718145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 164/335] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:27.723945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c2067-b7bc-468a-a5be-640f45ac51dc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:27.724433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c2067-b7bc-468a-a5be-640f45ac51dc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:27.724505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c2067-b7bc-468a-a5be-640f45ac51dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:27.744842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e10c2067-b7bc-468a-a5be-640f45ac51dc None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:27.746121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e10c2067-b7bc-468a-a5be-640f45ac51dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:27.842282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 172/336] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:27.874213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661117e-a870-4581-89ee-481aff132573 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:28.023858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661117e-a870-4581-89ee-481aff132573 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:28.071689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c983ef-adb3-4b70-9572-54f08b8ea213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:28.072510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c983ef-adb3-4b70-9572-54f08b8ea213 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:28.072797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c983ef-adb3-4b70-9572-54f08b8ea213 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:28.073066 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c983ef-adb3-4b70-9572-54f08b8ea213 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:28.084983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c983ef-adb3-4b70-9572-54f08b8ea213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f1c983ef-adb3-4b70-9572-54f08b8ea213 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:28.097531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:28.098500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:28.099283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 165/337] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:28.104995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c2067-b7bc-468a-a5be-640f45ac51dc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:28.134854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:28.234523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c2067-b7bc-468a-a5be-640f45ac51dc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:28.256949 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a888b-9c5d-4e39-af32-b050f76c8add None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:28.257247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a888b-9c5d-4e39-af32-b050f76c8add None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:28.257507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a888b-9c5d-4e39-af32-b050f76c8add None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:28.259177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 166/338] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:28.264326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35171ebf-5996-467f-ae97-be5a91f70f99 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:28.264615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35171ebf-5996-467f-ae97-be5a91f70f99 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:28.264896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35171ebf-5996-467f-ae97-be5a91f70f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:28.292456 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0a206-289c-43a4-b6a5-609cf8e24f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:28.293812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0a206-289c-43a4-b6a5-609cf8e24f17 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:28.294120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0a206-289c-43a4-b6a5-609cf8e24f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:28.294406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0a206-289c-43a4-b6a5-609cf8e24f17 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:28.295268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0a206-289c-43a4-b6a5-609cf8e24f17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:28.295577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0a206-289c-43a4-b6a5-609cf8e24f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:28.296969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0a206-289c-43a4-b6a5-609cf8e24f17 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:28.299094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-35171ebf-5996-467f-ae97-be5a91f70f99 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:28.300311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-35171ebf-5996-467f-ae97-be5a91f70f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:28.315584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 173/339] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:28.322159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0a206-289c-43a4-b6a5-609cf8e24f17 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:28.409490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:28.454436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 167/340] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:28.490384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:28.494531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35171ebf-5996-467f-ae97-be5a91f70f99 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:28.530513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0a206-289c-43a4-b6a5-609cf8e24f17 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:28.625625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ee9a4-6d02-4e86-8055-8924d517e23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:28.626251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ee9a4-6d02-4e86-8055-8924d517e23f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:28.626533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ee9a4-6d02-4e86-8055-8924d517e23f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:28.626822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ee9a4-6d02-4e86-8055-8924d517e23f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:28.679549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35171ebf-5996-467f-ae97-be5a91f70f99 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:28.728499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ee9a4-6d02-4e86-8055-8924d517e23f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:28.729975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ee9a4-6d02-4e86-8055-8924d517e23f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:28.757731 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c66e20-66b7-4d2d-a3ba-233bfe6947b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:28.758590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c66e20-66b7-4d2d-a3ba-233bfe6947b3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:28.758871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c66e20-66b7-4d2d-a3ba-233bfe6947b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:28.759147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c66e20-66b7-4d2d-a3ba-233bfe6947b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:28.784738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c66e20-66b7-4d2d-a3ba-233bfe6947b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-17c66e20-66b7-4d2d-a3ba-233bfe6947b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:28.798305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:28.799196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:28.800427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 168/341] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:28.807037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c66e20-66b7-4d2d-a3ba-233bfe6947b3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:28.830042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:28.855522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 174/342] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 10:51:28.857273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469ea59-108d-41be-8176-ff3b2343f125 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:28.857825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469ea59-108d-41be-8176-ff3b2343f125 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:28.858134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469ea59-108d-41be-8176-ff3b2343f125 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:28.859844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 175/343] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:28.862113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469ea59-108d-41be-8176-ff3b2343f125 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:28.881137 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:28.980822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c66e20-66b7-4d2d-a3ba-233bfe6947b3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:29.044475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469ea59-108d-41be-8176-ff3b2343f125 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:29.052566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705dc8c-86cb-44fb-bfc7-f11094342bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:29.053252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705dc8c-86cb-44fb-bfc7-f11094342bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.053532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705dc8c-86cb-44fb-bfc7-f11094342bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.053806 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705dc8c-86cb-44fb-bfc7-f11094342bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.054600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705dc8c-86cb-44fb-bfc7-f11094342bd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:29.054938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705dc8c-86cb-44fb-bfc7-f11094342bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:29.056375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705dc8c-86cb-44fb-bfc7-f11094342bd3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:29.066549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 169/344] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:29.067833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056e5f8-cd24-4898-b9aa-fb55b3792cf7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.068112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056e5f8-cd24-4898-b9aa-fb55b3792cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.068387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056e5f8-cd24-4898-b9aa-fb55b3792cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.086235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a056e5f8-cd24-4898-b9aa-fb55b3792cf7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:29.087429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a056e5f8-cd24-4898-b9aa-fb55b3792cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:29.091539 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0691eaa-68ef-4c1f-967b-212e82f9a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:29.092434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0691eaa-68ef-4c1f-967b-212e82f9a857 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.092715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0691eaa-68ef-4c1f-967b-212e82f9a857 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.092987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0691eaa-68ef-4c1f-967b-212e82f9a857 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.094215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0691eaa-68ef-4c1f-967b-212e82f9a857 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:29.095802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0691eaa-68ef-4c1f-967b-212e82f9a857 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:29.101968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 176/345] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:28 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:29.102935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0691eaa-68ef-4c1f-967b-212e82f9a857 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:29.161875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 170/346] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:29.162723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056e5f8-cd24-4898-b9aa-fb55b3792cf7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:29.194500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:29.199026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:29.260404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0691eaa-68ef-4c1f-967b-212e82f9a857 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:29.271197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056e5f8-cd24-4898-b9aa-fb55b3792cf7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:29.307682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4642b7c-a69b-4779-a398-2c1f86501d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:29.308758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4642b7c-a69b-4779-a398-2c1f86501d0e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.309044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4642b7c-a69b-4779-a398-2c1f86501d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.309316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4642b7c-a69b-4779-a398-2c1f86501d0e None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.311379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f9b39-ade7-4a00-a83e-a442b4cd62e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:29.312136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9b39-ade7-4a00-a83e-a442b4cd62e7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.312423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9b39-ade7-4a00-a83e-a442b4cd62e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.312697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f9b39-ade7-4a00-a83e-a442b4cd62e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.316083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4642b7c-a69b-4779-a398-2c1f86501d0e None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:29.317555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4642b7c-a69b-4779-a398-2c1f86501d0e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:29.322793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 171/347] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:29 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:29.322793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4642b7c-a69b-4779-a398-2c1f86501d0e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:29.398669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f9b39-ade7-4a00-a83e-a442b4cd62e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:29.400191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f9b39-ade7-4a00-a83e-a442b4cd62e7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:29.432896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:29.437779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4642b7c-a69b-4779-a398-2c1f86501d0e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:29.462500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:29.476581 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7da80-6202-4ce2-853f-2938729680c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:29.477403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7da80-6202-4ce2-853f-2938729680c1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.477684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7da80-6202-4ce2-853f-2938729680c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.477962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7da80-6202-4ce2-853f-2938729680c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.489662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7da80-6202-4ce2-853f-2938729680c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-58f7da80-6202-4ce2-853f-2938729680c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:29.497380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:29.499531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 172/348] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:29.501526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2db769-246e-4701-a008-dbe0d0ed7e09 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.503322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2db769-246e-4701-a008-dbe0d0ed7e09 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.503322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2db769-246e-4701-a008-dbe0d0ed7e09 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.503322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 173/349] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:29.503322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2db769-246e-4701-a008-dbe0d0ed7e09 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:29.526666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 177/350] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:29.528272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ab09-91ec-4a3e-a37c-252010465ee7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.528567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ab09-91ec-4a3e-a37c-252010465ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.528855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ab09-91ec-4a3e-a37c-252010465ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.547868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0902ab09-91ec-4a3e-a37c-252010465ee7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:29.549159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0902ab09-91ec-4a3e-a37c-252010465ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:29.614558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2db769-246e-4701-a008-dbe0d0ed7e09 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:29.624121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 178/351] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:29.624619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ab09-91ec-4a3e-a37c-252010465ee7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:29.650753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797a8c2-f00b-4992-9454-aa1a4db03bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:29.651421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a8c2-f00b-4992-9454-aa1a4db03bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.651698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a8c2-f00b-4992-9454-aa1a4db03bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.652032 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797a8c2-f00b-4992-9454-aa1a4db03bd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.652968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797a8c2-f00b-4992-9454-aa1a4db03bd0 None admin] RBAC: Adding query filter params (group.id=33f54fef6ed4400182390465adcd8a16, role.id=3e46d43e30ef494087da2c44f7ca702f, scope.project.id=9dd8fde445f5496fab61c82990f8ba09) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:29.653283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797a8c2-f00b-4992-9454-aa1a4db03bd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:29.654779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5797a8c2-f00b-4992-9454-aa1a4db03bd0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:29.660747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 174/352] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jun 20 10:51:29 2022] GET /identity/v3/role_assignments?group.id=33f54fef6ed4400182390465adcd8a16&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:29.661557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797a8c2-f00b-4992-9454-aa1a4db03bd0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:29.688122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:29.745622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:29.777331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ab09-91ec-4a3e-a37c-252010465ee7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:29.795334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5797a8c2-f00b-4992-9454-aa1a4db03bd0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:29.821637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0e7a2-16f6-48bb-8b91-4c49e99c62a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:29.822376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0e7a2-16f6-48bb-8b91-4c49e99c62a7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.822657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0e7a2-16f6-48bb-8b91-4c49e99c62a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.822928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0e7a2-16f6-48bb-8b91-4c49e99c62a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.823846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0e7a2-16f6-48bb-8b91-4c49e99c62a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:29.824221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0e7a2-16f6-48bb-8b91-4c49e99c62a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:29.825690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0e7a2-16f6-48bb-8b91-4c49e99c62a7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:29.838600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 179/353] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:29.839524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0e7a2-16f6-48bb-8b91-4c49e99c62a7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:29.845186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654d384-cd98-4c7b-9f41-7b79724eb50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:29.846127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654d384-cd98-4c7b-9f41-7b79724eb50b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.846397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654d384-cd98-4c7b-9f41-7b79724eb50b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.846680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654d384-cd98-4c7b-9f41-7b79724eb50b None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.849884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654d384-cd98-4c7b-9f41-7b79724eb50b None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:29.851407 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654d384-cd98-4c7b-9f41-7b79724eb50b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:29.853216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 175/354] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:29 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:29.855690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b048ed-ede8-422d-8c6c-fb1a81519e3c None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:29.855988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b048ed-ede8-422d-8c6c-fb1a81519e3c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:29.856267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b048ed-ede8-422d-8c6c-fb1a81519e3c None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:29.857905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 176/355] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:29.858862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b048ed-ede8-422d-8c6c-fb1a81519e3c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:29.975157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:29.977577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0e7a2-16f6-48bb-8b91-4c49e99c62a7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:29.992060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b048ed-ede8-422d-8c6c-fb1a81519e3c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:30.022790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411ba1b-d9c5-487e-bfed-0a7ef4f3a2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:30.023622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411ba1b-d9c5-487e-bfed-0a7ef4f3a2db None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.023900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411ba1b-d9c5-487e-bfed-0a7ef4f3a2db None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.024169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411ba1b-d9c5-487e-bfed-0a7ef4f3a2db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.034681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:30.038848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad6bbc-da98-411f-bc83-aa227bef5954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:30.039141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411ba1b-d9c5-487e-bfed-0a7ef4f3a2db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:30.039517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad6bbc-da98-411f-bc83-aa227bef5954 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.039800 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad6bbc-da98-411f-bc83-aa227bef5954 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.040060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad6bbc-da98-411f-bc83-aa227bef5954 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e411ba1b-d9c5-487e-bfed-0a7ef4f3a2db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:30.049462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:30.050652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:30.051446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:30.052051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 180/356] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:30.057064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411ba1b-d9c5-487e-bfed-0a7ef4f3a2db None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:30.125173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad6bbc-da98-411f-bc83-aa227bef5954 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:30.127088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad6bbc-da98-411f-bc83-aa227bef5954 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:30.182339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411ba1b-d9c5-487e-bfed-0a7ef4f3a2db None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:30.230654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0687fdb-4f9a-4aac-888e-c84dffa57f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:30.231464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0687fdb-4f9a-4aac-888e-c84dffa57f96 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.231745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0687fdb-4f9a-4aac-888e-c84dffa57f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.232022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0687fdb-4f9a-4aac-888e-c84dffa57f96 None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.234495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0687fdb-4f9a-4aac-888e-c84dffa57f96 None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:30.237272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0687fdb-4f9a-4aac-888e-c84dffa57f96 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:30.237809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:30.238650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 181/357] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:30 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:30.242179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673c6e-2c51-480f-9c87-0f58270eeb04 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.242179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673c6e-2c51-480f-9c87-0f58270eeb04 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.242179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673c6e-2c51-480f-9c87-0f58270eeb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.248636 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 177/358] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:30.251972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79924e9-6286-4685-8e2d-05d86cb68264 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.252436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79924e9-6286-4685-8e2d-05d86cb68264 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.252955 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79924e9-6286-4685-8e2d-05d86cb68264 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.254631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 178/359] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:30.255356 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79924e9-6286-4685-8e2d-05d86cb68264 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:30.264842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-76673c6e-2c51-480f-9c87-0f58270eeb04 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:30.266985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-76673c6e-2c51-480f-9c87-0f58270eeb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:30.355725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:30.362526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 182/360] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:30.363381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76673c6e-2c51-480f-9c87-0f58270eeb04 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:30.383339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79924e9-6286-4685-8e2d-05d86cb68264 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:30.431828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d5ccb-c163-4c56-ad4f-407bf00f2add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:30.432843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d5ccb-c163-4c56-ad4f-407bf00f2add None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.433380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d5ccb-c163-4c56-ad4f-407bf00f2add None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.434034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d5ccb-c163-4c56-ad4f-407bf00f2add None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.434987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d5ccb-c163-4c56-ad4f-407bf00f2add None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:30.435925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d5ccb-c163-4c56-ad4f-407bf00f2add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:30.436765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d5ccb-c163-4c56-ad4f-407bf00f2add None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:30.447053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 179/361] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:30.451914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d5ccb-c163-4c56-ad4f-407bf00f2add None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:30.499067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76673c6e-2c51-480f-9c87-0f58270eeb04 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:30.517045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:30.543861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe1d42-32eb-46e7-b527-34f9b76fcd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:30.544531 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe1d42-32eb-46e7-b527-34f9b76fcd23 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.544810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe1d42-32eb-46e7-b527-34f9b76fcd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.545081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe1d42-32eb-46e7-b527-34f9b76fcd23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.545918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe1d42-32eb-46e7-b527-34f9b76fcd23 None admin] RBAC: Adding query filter params (group.id=33f54fef6ed4400182390465adcd8a16, role.id=ffab93457c0f4f64a29764f17759396e, scope.project.id=9dd8fde445f5496fab61c82990f8ba09) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:30.546260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe1d42-32eb-46e7-b527-34f9b76fcd23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:30.547701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe1d42-32eb-46e7-b527-34f9b76fcd23 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:30.553578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 183/362] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jun 20 10:51:30 2022] GET /identity/v3/role_assignments?group.id=33f54fef6ed4400182390465adcd8a16&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=ffab93457c0f4f64a29764f17759396e => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:30.555599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a04d9-121a-4da5-9921-919b1bebbf41 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.555884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a04d9-121a-4da5-9921-919b1bebbf41 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.556154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a04d9-121a-4da5-9921-919b1bebbf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.576459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ec4a04d9-121a-4da5-9921-919b1bebbf41 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:30.577284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d5ccb-c163-4c56-ad4f-407bf00f2add None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:30.577677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ec4a04d9-121a-4da5-9921-919b1bebbf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:30.628927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73710856-46ab-4bed-ae05-438f32347978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:30.630238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73710856-46ab-4bed-ae05-438f32347978 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.630712 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73710856-46ab-4bed-ae05-438f32347978 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.631165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73710856-46ab-4bed-ae05-438f32347978 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.633545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73710856-46ab-4bed-ae05-438f32347978 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:30.635357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73710856-46ab-4bed-ae05-438f32347978 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:30.638276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 180/363] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:30 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:30.639426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73710856-46ab-4bed-ae05-438f32347978 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:30.660461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:30.675231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 184/364] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:30.676348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a04d9-121a-4da5-9921-919b1bebbf41 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:30.786857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73710856-46ab-4bed-ae05-438f32347978 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:30.801629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a04d9-121a-4da5-9921-919b1bebbf41 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:30.815721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:30.833516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b45138-e85c-48a6-8dce-4406b85a31d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:30.834999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b45138-e85c-48a6-8dce-4406b85a31d3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.835284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b45138-e85c-48a6-8dce-4406b85a31d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.835562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b45138-e85c-48a6-8dce-4406b85a31d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.851177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b45138-e85c-48a6-8dce-4406b85a31d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:30.851550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa59d36-8638-4b1a-acaa-8d451a26a007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:30.852424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa59d36-8638-4b1a-acaa-8d451a26a007 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:30.852911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa59d36-8638-4b1a-acaa-8d451a26a007 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:30.853416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa59d36-8638-4b1a-acaa-8d451a26a007 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-85b45138-e85c-48a6-8dce-4406b85a31d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:30.861698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:30.862683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 10:51:30.863462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:30.866369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 181/365] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:30.867485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b45138-e85c-48a6-8dce-4406b85a31d3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:30.930439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:30.946431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa59d36-8638-4b1a-acaa-8d451a26a007 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:30.948035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa59d36-8638-4b1a-acaa-8d451a26a007 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:30.995717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b45138-e85c-48a6-8dce-4406b85a31d3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:31.041380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194c7e4-b176-4da9-94be-aa00a5c7371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:31.042259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194c7e4-b176-4da9-94be-aa00a5c7371e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:31.042547 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194c7e4-b176-4da9-94be-aa00a5c7371e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:31.042915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194c7e4-b176-4da9-94be-aa00a5c7371e None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:31.044130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194c7e4-b176-4da9-94be-aa00a5c7371e None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:31.045605 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194c7e4-b176-4da9-94be-aa00a5c7371e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:31.047931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 182/366] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:30 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:31.048717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194c7e4-b176-4da9-94be-aa00a5c7371e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:31.080950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 185/367] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:31.081912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa59d36-8638-4b1a-acaa-8d451a26a007 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:31.109200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:31.164820 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194c7e4-b176-4da9-94be-aa00a5c7371e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:31.180220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:31.190067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa59d36-8638-4b1a-acaa-8d451a26a007 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:31.201635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e1ba8-7b50-49cb-baca-068dc5975298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:31.202582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e1ba8-7b50-49cb-baca-068dc5975298 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:31.202870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e1ba8-7b50-49cb-baca-068dc5975298 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:31.203150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e1ba8-7b50-49cb-baca-068dc5975298 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:31.215484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e1ba8-7b50-49cb-baca-068dc5975298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5e5e1ba8-7b50-49cb-baca-068dc5975298 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:31.224864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:31.225750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:31.226558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:31.227715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 183/368] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:51:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:31.227715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e1ba8-7b50-49cb-baca-068dc5975298 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:31.230947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1441b98-eb29-46f3-af9e-ba3b19bc2409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:31.231643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1441b98-eb29-46f3-af9e-ba3b19bc2409 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:31.231927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1441b98-eb29-46f3-af9e-ba3b19bc2409 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:31.232209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1441b98-eb29-46f3-af9e-ba3b19bc2409 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:31.232975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1441b98-eb29-46f3-af9e-ba3b19bc2409 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:31.233285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1441b98-eb29-46f3-af9e-ba3b19bc2409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:31.234712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1441b98-eb29-46f3-af9e-ba3b19bc2409 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:31.246303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 186/369] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:31.247161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1441b98-eb29-46f3-af9e-ba3b19bc2409 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:31.338539 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e1ba8-7b50-49cb-baca-068dc5975298 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:31.348786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:31.349037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1441b98-eb29-46f3-af9e-ba3b19bc2409 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:31.378989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3321e-243b-4315-a099-1bc34e8294b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:31.379760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3321e-243b-4315-a099-1bc34e8294b8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:31.380113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3321e-243b-4315-a099-1bc34e8294b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:31.380395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3321e-243b-4315-a099-1bc34e8294b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:31.381241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3321e-243b-4315-a099-1bc34e8294b8 None admin] RBAC: Adding query filter params (group.id=33f54fef6ed4400182390465adcd8a16, role.id=3e46d43e30ef494087da2c44f7ca702f, scope.project.id=205237c47ce44b8b839b43198fb4c1f1) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:31.381554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3321e-243b-4315-a099-1bc34e8294b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:31.383357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3321e-243b-4315-a099-1bc34e8294b8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:31.384629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748e333-bd24-4928-b93f-eaf1fda180d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:31.385510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748e333-bd24-4928-b93f-eaf1fda180d7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:31.385797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748e333-bd24-4928-b93f-eaf1fda180d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:31.386074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748e333-bd24-4928-b93f-eaf1fda180d7 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:31.387695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748e333-bd24-4928-b93f-eaf1fda180d7 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:31.389257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748e333-bd24-4928-b93f-eaf1fda180d7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:31.389697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 184/370] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jun 20 10:51:31 2022] GET /identity/v3/role_assignments?group.id=33f54fef6ed4400182390465adcd8a16&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:31.390534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3321e-243b-4315-a099-1bc34e8294b8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:31.391426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 187/371] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:31 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:31.392201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748e333-bd24-4928-b93f-eaf1fda180d7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:31.419644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:31.514884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3321e-243b-4315-a099-1bc34e8294b8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:31.524637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748e333-bd24-4928-b93f-eaf1fda180d7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:31.555504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c230cc-7f6d-4707-9180-dcecd1884365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:31.556203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c230cc-7f6d-4707-9180-dcecd1884365 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:31.556485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c230cc-7f6d-4707-9180-dcecd1884365 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:31.556763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c230cc-7f6d-4707-9180-dcecd1884365 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:31.557532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c230cc-7f6d-4707-9180-dcecd1884365 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:31.557843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c230cc-7f6d-4707-9180-dcecd1884365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:31.559576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c230cc-7f6d-4707-9180-dcecd1884365 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:31.569365 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 185/372] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:31.570378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c230cc-7f6d-4707-9180-dcecd1884365 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:31.583866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e9f30-e193-4847-b06b-30e3e7ad6764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:31.584789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9f30-e193-4847-b06b-30e3e7ad6764 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:31.585076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9f30-e193-4847-b06b-30e3e7ad6764 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:31.585354 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9f30-e193-4847-b06b-30e3e7ad6764 None admin] PATH_INFO: `/v3/users/b467f816224740e5a2e9dc6a8736dd24` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:31.591149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e9f30-e193-4847-b06b-30e3e7ad6764 None admin] RBAC: Authorizing `identity:get_user(user_id=b467f816224740e5a2e9dc6a8736dd24)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:31.592711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e9f30-e193-4847-b06b-30e3e7ad6764 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:31.594825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 188/373] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:51:31 2022] GET /identity/v3/users/b467f816224740e5a2e9dc6a8736dd24 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:31.595916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e9f30-e193-4847-b06b-30e3e7ad6764 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:31.610689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:31.714873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c230cc-7f6d-4707-9180-dcecd1884365 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:31.745509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:31.759797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e9f30-e193-4847-b06b-30e3e7ad6764 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:31.766149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c308e-a367-452b-9378-90037eaaea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:31.767035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c308e-a367-452b-9378-90037eaaea65 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:31.767318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c308e-a367-452b-9378-90037eaaea65 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:31.767594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c308e-a367-452b-9378-90037eaaea65 None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:31.769336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c308e-a367-452b-9378-90037eaaea65 None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:31.771331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c308e-a367-452b-9378-90037eaaea65 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:31.773725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 186/374] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:31 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:31.774518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c308e-a367-452b-9378-90037eaaea65 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:31.804921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72696b9-737a-4666-8d32-0fe57fca8b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:31.805622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72696b9-737a-4666-8d32-0fe57fca8b4d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:31.806424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72696b9-737a-4666-8d32-0fe57fca8b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:31.806701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72696b9-737a-4666-8d32-0fe57fca8b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:31.917087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c308e-a367-452b-9378-90037eaaea65 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:31.917649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:31.955566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1fb28-e097-41dd-8128-68ce4b3e0124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:31.956627 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1fb28-e097-41dd-8128-68ce4b3e0124 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:31.956912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1fb28-e097-41dd-8128-68ce4b3e0124 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:31.957189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1fb28-e097-41dd-8128-68ce4b3e0124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:31.958399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1fb28-e097-41dd-8128-68ce4b3e0124 None admin] RBAC: Adding query filter params (role.id=e4214ef8456541528194f90e229b356a, user.id=b467f816224740e5a2e9dc6a8736dd24, scope.system=all) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:31.958728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1fb28-e097-41dd-8128-68ce4b3e0124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:31.960483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1fb28-e097-41dd-8128-68ce4b3e0124 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:31.982293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 187/375] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Jun 20 10:51:31 2022] GET /identity/v3/role_assignments?user.id=b467f816224740e5a2e9dc6a8736dd24&scope.system=all&role.id=e4214ef8456541528194f90e229b356a => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:31.983138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1fb28-e097-41dd-8128-68ce4b3e0124 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:31.986163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72696b9-737a-4666-8d32-0fe57fca8b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:31.987741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72696b9-737a-4666-8d32-0fe57fca8b4d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:32.107614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:32.120424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 189/376] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:32.126234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72696b9-737a-4666-8d32-0fe57fca8b4d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:32.167877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1fb28-e097-41dd-8128-68ce4b3e0124 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:32.227728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092b1e1-66f9-4b35-b2f9-30bb86fd5623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:32.228763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092b1e1-66f9-4b35-b2f9-30bb86fd5623 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:32.229233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092b1e1-66f9-4b35-b2f9-30bb86fd5623 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:32.229687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092b1e1-66f9-4b35-b2f9-30bb86fd5623 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:32.230831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092b1e1-66f9-4b35-b2f9-30bb86fd5623 None admin] RBAC: Adding query filter params (group.id=33f54fef6ed4400182390465adcd8a16, role.id=ffab93457c0f4f64a29764f17759396e, scope.project.id=205237c47ce44b8b839b43198fb4c1f1) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:32.231360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092b1e1-66f9-4b35-b2f9-30bb86fd5623 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:32.233047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092b1e1-66f9-4b35-b2f9-30bb86fd5623 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:32.244543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72696b9-737a-4666-8d32-0fe57fca8b4d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:32.247591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 188/377] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jun 20 10:51:31 2022] GET /identity/v3/role_assignments?group.id=33f54fef6ed4400182390465adcd8a16&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=ffab93457c0f4f64a29764f17759396e => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:32.284028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27cb70a-1450-4e69-9960-1f89d5d8e879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:32.284961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cb70a-1450-4e69-9960-1f89d5d8e879 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:32.285246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cb70a-1450-4e69-9960-1f89d5d8e879 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:32.285521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27cb70a-1450-4e69-9960-1f89d5d8e879 None admin] PATH_INFO: `/v3/projects/f76a7fac220343628fdd3080ad89317e` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:32.286781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27cb70a-1450-4e69-9960-1f89d5d8e879 None admin] RBAC: Authorizing `identity:get_project(project_id=f76a7fac220343628fdd3080ad89317e)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:32.288238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27cb70a-1450-4e69-9960-1f89d5d8e879 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:32.290481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 190/378] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:32 2022] GET /identity/v3/projects/f76a7fac220343628fdd3080ad89317e => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:32.296351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092b1e1-66f9-4b35-b2f9-30bb86fd5623 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:32.326820 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:32.423209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad9fd4-d875-41f1-af6e-9f54c534ee21 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:32.423514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad9fd4-d875-41f1-af6e-9f54c534ee21 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:32.423676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad9fd4-d875-41f1-af6e-9f54c534ee21 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:32.425071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 191/379] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:51:32.434681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092b1e1-66f9-4b35-b2f9-30bb86fd5623 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:32.440844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15efc0-a39d-4e96-9ab6-78830fb08fec None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:32.441143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15efc0-a39d-4e96-9ab6-78830fb08fec None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:32.441438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15efc0-a39d-4e96-9ab6-78830fb08fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:32.475285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-fc15efc0-a39d-4e96-9ab6-78830fb08fec None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:32.476526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fc15efc0-a39d-4e96-9ab6-78830fb08fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:32.483842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418418be-489d-49fd-83ab-125b0796149a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:32.484729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418418be-489d-49fd-83ab-125b0796149a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:32.485010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418418be-489d-49fd-83ab-125b0796149a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:32.485284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418418be-489d-49fd-83ab-125b0796149a None admin] PATH_INFO: `/v3/groups/69f8119ecb9443f0bd5e994f16671787` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:32.499585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418418be-489d-49fd-83ab-125b0796149a None admin] RBAC: Authorizing `identity:get_group(group_id=69f8119ecb9443f0bd5e994f16671787)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:32.501064 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418418be-489d-49fd-83ab-125b0796149a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:32.503140 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 189/380] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:32 2022] GET /identity/v3/groups/69f8119ecb9443f0bd5e994f16671787 => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:32.511042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418418be-489d-49fd-83ab-125b0796149a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:32.551295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:32.568124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 192/381] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:32.609829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15efc0-a39d-4e96-9ab6-78830fb08fec None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:32.679843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:32.706710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418418be-489d-49fd-83ab-125b0796149a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:32.779617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246cd35-f6e0-4763-8634-551fe7e725d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:32.780296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246cd35-f6e0-4763-8634-551fe7e725d2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:32.780582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246cd35-f6e0-4763-8634-551fe7e725d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:32.780866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246cd35-f6e0-4763-8634-551fe7e725d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:32.781855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246cd35-f6e0-4763-8634-551fe7e725d2 None admin] RBAC: Adding query filter params (group.id=69f8119ecb9443f0bd5e994f16671787, role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=f76a7fac220343628fdd3080ad89317e) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:32.782188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246cd35-f6e0-4763-8634-551fe7e725d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:32.783665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246cd35-f6e0-4763-8634-551fe7e725d2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:32.794943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15efc0-a39d-4e96-9ab6-78830fb08fec None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:32.801205 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 190/382] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jun 20 10:51:32 2022] GET /identity/v3/role_assignments?group.id=69f8119ecb9443f0bd5e994f16671787&scope.project.id=f76a7fac220343628fdd3080ad89317e&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 260 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:32.836024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f033b-13f4-40b5-815b-1acbbfc0bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:32.836843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f033b-13f4-40b5-815b-1acbbfc0bd4c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:32.837129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f033b-13f4-40b5-815b-1acbbfc0bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:32.837404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f033b-13f4-40b5-815b-1acbbfc0bd4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:32.849150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f033b-13f4-40b5-815b-1acbbfc0bd4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-f43f033b-13f4-40b5-815b-1acbbfc0bd4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:32.857625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:32.858651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:32.859598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 193/383] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:32.867095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246cd35-f6e0-4763-8634-551fe7e725d2 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:33.002078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246cd35-f6e0-4763-8634-551fe7e725d2 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:33.052591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06478934-c923-46d6-afb7-e3f5e98b9043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:33.053646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06478934-c923-46d6-afb7-e3f5e98b9043 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:33.054130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06478934-c923-46d6-afb7-e3f5e98b9043 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:33.054586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06478934-c923-46d6-afb7-e3f5e98b9043 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:33.055587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06478934-c923-46d6-afb7-e3f5e98b9043 None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:33.056088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06478934-c923-46d6-afb7-e3f5e98b9043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:33.057739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06478934-c923-46d6-afb7-e3f5e98b9043 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:33.073438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 191/384] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:33.075288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f033b-13f4-40b5-815b-1acbbfc0bd4c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:33.136511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:33.195059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f033b-13f4-40b5-815b-1acbbfc0bd4c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:33.238136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ec69e-45bd-48ac-b0f1-74545c4ad844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:33.239036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ec69e-45bd-48ac-b0f1-74545c4ad844 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:33.239326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ec69e-45bd-48ac-b0f1-74545c4ad844 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:33.239602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ec69e-45bd-48ac-b0f1-74545c4ad844 None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:33.240806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ec69e-45bd-48ac-b0f1-74545c4ad844 None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:33.242309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ec69e-45bd-48ac-b0f1-74545c4ad844 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:33.245263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 194/385] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:33 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:33.250450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06478934-c923-46d6-afb7-e3f5e98b9043 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:33.279322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:33.409723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06478934-c923-46d6-afb7-e3f5e98b9043 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:33.450946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d416226-794c-40a5-aa93-5d6b21b78524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:33.451822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d416226-794c-40a5-aa93-5d6b21b78524 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:33.452107 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d416226-794c-40a5-aa93-5d6b21b78524 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:33.452382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d416226-794c-40a5-aa93-5d6b21b78524 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:33.453593 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d416226-794c-40a5-aa93-5d6b21b78524 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:33.455084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d416226-794c-40a5-aa93-5d6b21b78524 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:33.457110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 192/386] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:33 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:33.463123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ec69e-45bd-48ac-b0f1-74545c4ad844 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:33.566774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287a71f-fd7b-40cd-b206-566f275abd66 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:33.567076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287a71f-fd7b-40cd-b206-566f275abd66 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:33.567254 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287a71f-fd7b-40cd-b206-566f275abd66 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:33.568656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 193/387] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:51:33.575150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c3cb7-2217-4dc7-a02a-381a7f88d7a7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:33.575427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c3cb7-2217-4dc7-a02a-381a7f88d7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:33.575706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c3cb7-2217-4dc7-a02a-381a7f88d7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:33.594622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-769c3cb7-2217-4dc7-a02a-381a7f88d7a7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:33.595937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-769c3cb7-2217-4dc7-a02a-381a7f88d7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:33.598538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ec69e-45bd-48ac-b0f1-74545c4ad844 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:33.619588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:33.652743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd8a4d-a810-448a-ba2a-52516861140e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:33.654058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd8a4d-a810-448a-ba2a-52516861140e None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:33.654581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd8a4d-a810-448a-ba2a-52516861140e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:33.655083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd8a4d-a810-448a-ba2a-52516861140e None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09/groups/33f54fef6ed4400182390465adcd8a16/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:33.658076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd8a4d-a810-448a-ba2a-52516861140e None admin] RBAC: Authorizing `identity:create_grant(project_id=9dd8fde445f5496fab61c82990f8ba09, group_id=33f54fef6ed4400182390465adcd8a16, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:33.659893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd8a4d-a810-448a-ba2a-52516861140e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:33.683784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 195/388] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jun 20 10:51:33 2022] PUT /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09/groups/33f54fef6ed4400182390465adcd8a16/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:33.707089 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 194/389] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:33.735115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd8a4d-a810-448a-ba2a-52516861140e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:33.875217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd8a4d-a810-448a-ba2a-52516861140e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:33.913636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3387-1c9e-4227-8799-b09093b4bce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:33.914505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3387-1c9e-4227-8799-b09093b4bce4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:33.914788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3387-1c9e-4227-8799-b09093b4bce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:33.915065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3387-1c9e-4227-8799-b09093b4bce4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:33.926274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc3387-1c9e-4227-8799-b09093b4bce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-8fbc3387-1c9e-4227-8799-b09093b4bce4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:33.934650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:33.935538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:33.936314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 196/390] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:33.943387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c3cb7-2217-4dc7-a02a-381a7f88d7a7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:33.959499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:34.050651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c3cb7-2217-4dc7-a02a-381a7f88d7a7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:34.094507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6a7c2-58ad-47e6-9415-4c074967d642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:34.095188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a7c2-58ad-47e6-9415-4c074967d642 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:34.095477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a7c2-58ad-47e6-9415-4c074967d642 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:34.095753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a7c2-58ad-47e6-9415-4c074967d642 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:34.096516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6a7c2-58ad-47e6-9415-4c074967d642 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:34.096827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6a7c2-58ad-47e6-9415-4c074967d642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:34.098273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6a7c2-58ad-47e6-9415-4c074967d642 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:34.110289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 195/391] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:34.115840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3387-1c9e-4227-8799-b09093b4bce4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:34.162636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:34.236117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3387-1c9e-4227-8799-b09093b4bce4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:34.270952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043183b-54bb-4c04-8806-9b9059f6f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:34.271834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043183b-54bb-4c04-8806-9b9059f6f275 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:34.272120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043183b-54bb-4c04-8806-9b9059f6f275 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:34.272398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043183b-54bb-4c04-8806-9b9059f6f275 None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:34.273588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043183b-54bb-4c04-8806-9b9059f6f275 None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:34.275088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043183b-54bb-4c04-8806-9b9059f6f275 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:34.277511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 197/392] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:34 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:34.283108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6a7c2-58ad-47e6-9415-4c074967d642 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:34.364785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:34.396161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6a7c2-58ad-47e6-9415-4c074967d642 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:34.411845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c534448-64b9-4876-ae19-c4c1739c4ee5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:34.412129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c534448-64b9-4876-ae19-c4c1739c4ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:34.412410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c534448-64b9-4876-ae19-c4c1739c4ee5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:34.413865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 198/393] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:51:34.423449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6d09c-8961-4180-a56c-76a3f7d2dc47 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:34.423738 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6d09c-8961-4180-a56c-76a3f7d2dc47 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:34.424019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6d09c-8961-4180-a56c-76a3f7d2dc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:34.446143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2be6d09c-8961-4180-a56c-76a3f7d2dc47 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:34.449095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2be6d09c-8961-4180-a56c-76a3f7d2dc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:34.471422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd16354-eb34-48cd-95fa-2dc4e95404e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:34.472369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd16354-eb34-48cd-95fa-2dc4e95404e6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:34.472697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd16354-eb34-48cd-95fa-2dc4e95404e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:34.472980 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd16354-eb34-48cd-95fa-2dc4e95404e6 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:34.476294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd16354-eb34-48cd-95fa-2dc4e95404e6 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:34.477839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd16354-eb34-48cd-95fa-2dc4e95404e6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:34.481617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 196/394] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:34 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:34.500051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd16354-eb34-48cd-95fa-2dc4e95404e6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:34.594125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 199/395] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:34.643214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6d09c-8961-4180-a56c-76a3f7d2dc47 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:34.659334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd16354-eb34-48cd-95fa-2dc4e95404e6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:34.697857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd89b70-eb0b-4fc9-a4f8-b2d49099babb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:34.699606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd89b70-eb0b-4fc9-a4f8-b2d49099babb None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:34.700104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd89b70-eb0b-4fc9-a4f8-b2d49099babb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:34.700570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd89b70-eb0b-4fc9-a4f8-b2d49099babb None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09/groups/33f54fef6ed4400182390465adcd8a16/roles/ffab93457c0f4f64a29764f17759396e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:34.702919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd89b70-eb0b-4fc9-a4f8-b2d49099babb None admin] RBAC: Authorizing `identity:create_grant(project_id=9dd8fde445f5496fab61c82990f8ba09, group_id=33f54fef6ed4400182390465adcd8a16, role_id=ffab93457c0f4f64a29764f17759396e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:34.704964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd89b70-eb0b-4fc9-a4f8-b2d49099babb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:34.721398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 197/396] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jun 20 10:51:34 2022] PUT /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09/groups/33f54fef6ed4400182390465adcd8a16/roles/ffab93457c0f4f64a29764f17759396e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:34.732924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:34.740506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a6c8c-3d23-4c2b-9384-6e5b1dc6b120 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:34.740833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a6c8c-3d23-4c2b-9384-6e5b1dc6b120 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:34.741073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a6c8c-3d23-4c2b-9384-6e5b1dc6b120 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:34.743060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 198/397] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:51:34.752074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1361a-a9a7-4a32-bde7-e2dc9f0329ec None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:34.752515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1361a-a9a7-4a32-bde7-e2dc9f0329ec None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:34.752893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1361a-a9a7-4a32-bde7-e2dc9f0329ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:34.759124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:34.774647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f6b1361a-a9a7-4a32-bde7-e2dc9f0329ec None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:34.776126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f6b1361a-a9a7-4a32-bde7-e2dc9f0329ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:34.862771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 199/398] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:34.890291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1361a-a9a7-4a32-bde7-e2dc9f0329ec None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:34.902725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6d09c-8961-4180-a56c-76a3f7d2dc47 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:34.956634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b70d7-50e9-4e8f-b56e-255e9a9962ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:34.957369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b70d7-50e9-4e8f-b56e-255e9a9962ea None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:34.957655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b70d7-50e9-4e8f-b56e-255e9a9962ea None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:34.957931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b70d7-50e9-4e8f-b56e-255e9a9962ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:34.973444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b70d7-50e9-4e8f-b56e-255e9a9962ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-5f4b70d7-50e9-4e8f-b56e-255e9a9962ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:34.983318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:34.984282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:34.985060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:34.985644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 200/399] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:34.991290 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b70d7-50e9-4e8f-b56e-255e9a9962ea None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:35.024013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1361a-a9a7-4a32-bde7-e2dc9f0329ec None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:35.059890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:35.075153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093aa3d0-41e4-47a8-a35f-012e1696b647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:35.075979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093aa3d0-41e4-47a8-a35f-012e1696b647 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:35.076255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093aa3d0-41e4-47a8-a35f-012e1696b647 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:35.076523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093aa3d0-41e4-47a8-a35f-012e1696b647 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:35.099427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093aa3d0-41e4-47a8-a35f-012e1696b647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-093aa3d0-41e4-47a8-a35f-012e1696b647 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:35.110003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:35.110891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:35.111680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:35.121678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 200/400] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:35.127282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093aa3d0-41e4-47a8-a35f-012e1696b647 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:35.184036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:35.194136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b70d7-50e9-4e8f-b56e-255e9a9962ea None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:35.243352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093aa3d0-41e4-47a8-a35f-012e1696b647 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:35.262971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ef9b8-8ec2-4b21-871d-bd11abb3dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:35.264027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ef9b8-8ec2-4b21-871d-bd11abb3dec6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:35.264530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ef9b8-8ec2-4b21-871d-bd11abb3dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:35.265006 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ef9b8-8ec2-4b21-871d-bd11abb3dec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:35.265954 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ef9b8-8ec2-4b21-871d-bd11abb3dec6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:35.267226 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ef9b8-8ec2-4b21-871d-bd11abb3dec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:35.269725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ef9b8-8ec2-4b21-871d-bd11abb3dec6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:35.287764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 201/401] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:35.295747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ef9b8-8ec2-4b21-871d-bd11abb3dec6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:35.315730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5185b-f2bd-4d41-ba3d-cfd7093ab87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:35.316799 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5185b-f2bd-4d41-ba3d-cfd7093ab87e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:35.317306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5185b-f2bd-4d41-ba3d-cfd7093ab87e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:35.317767 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5185b-f2bd-4d41-ba3d-cfd7093ab87e None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:35.318804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5185b-f2bd-4d41-ba3d-cfd7093ab87e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:35.319381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5185b-f2bd-4d41-ba3d-cfd7093ab87e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:35.322323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5185b-f2bd-4d41-ba3d-cfd7093ab87e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:35.340465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 201/402] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:35.346316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5185b-f2bd-4d41-ba3d-cfd7093ab87e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:35.449676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ef9b8-8ec2-4b21-871d-bd11abb3dec6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:35.454440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:35.502894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbd078-9f2b-4097-9f34-a862d4525b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:35.503702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbd078-9f2b-4097-9f34-a862d4525b17 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:35.503984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbd078-9f2b-4097-9f34-a862d4525b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:35.504258 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbd078-9f2b-4097-9f34-a862d4525b17 None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:35.505452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbd078-9f2b-4097-9f34-a862d4525b17 None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:35.506930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbd078-9f2b-4097-9f34-a862d4525b17 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:35.508879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 202/403] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:35 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:35.514400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5185b-f2bd-4d41-ba3d-cfd7093ab87e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:35.515483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbd078-9f2b-4097-9f34-a862d4525b17 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:35.562380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0714c-0a7b-4ab7-9397-d97f0f9674d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:35.563132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:35.564922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0714c-0a7b-4ab7-9397-d97f0f9674d6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:35.566122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0714c-0a7b-4ab7-9397-d97f0f9674d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:35.568545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0714c-0a7b-4ab7-9397-d97f0f9674d6 None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:35.569961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0714c-0a7b-4ab7-9397-d97f0f9674d6 None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:35.572355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0714c-0a7b-4ab7-9397-d97f0f9674d6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:35.577613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 202/404] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:35 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:35.583344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0714c-0a7b-4ab7-9397-d97f0f9674d6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:35.690694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbd078-9f2b-4097-9f34-a862d4525b17 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:35.723367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0714c-0a7b-4ab7-9397-d97f0f9674d6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:35.738831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106d189-9c10-4cf0-a24a-84ab26b5bec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:35.740329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106d189-9c10-4cf0-a24a-84ab26b5bec9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:35.740957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106d189-9c10-4cf0-a24a-84ab26b5bec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:35.741461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106d189-9c10-4cf0-a24a-84ab26b5bec9 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:35.743105 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106d189-9c10-4cf0-a24a-84ab26b5bec9 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:35.745359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106d189-9c10-4cf0-a24a-84ab26b5bec9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:35.748167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 203/405] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:35 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:35.749544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:35.755036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e41aa-9dfc-4454-8971-178a700a90d2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:35.755321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e41aa-9dfc-4454-8971-178a700a90d2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:35.755659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e41aa-9dfc-4454-8971-178a700a90d2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:35.762120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 204/406] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:35.762120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e41aa-9dfc-4454-8971-178a700a90d2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:35.770912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f7b96-a582-4e58-a11f-6d3ebf6de247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:35.771805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f7b96-a582-4e58-a11f-6d3ebf6de247 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:35.772679 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f7b96-a582-4e58-a11f-6d3ebf6de247 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:35.773017 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f7b96-a582-4e58-a11f-6d3ebf6de247 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:35.774302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f7b96-a582-4e58-a11f-6d3ebf6de247 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:35.776229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f7b96-a582-4e58-a11f-6d3ebf6de247 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:35.782128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 203/407] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:35 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:35.782128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ce8eb-2477-48e8-aebd-ea77dbaace58 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:35.782128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ce8eb-2477-48e8-aebd-ea77dbaace58 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:35.782128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ce8eb-2477-48e8-aebd-ea77dbaace58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:35.800284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-799ce8eb-2477-48e8-aebd-ea77dbaace58 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:35.801643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-799ce8eb-2477-48e8-aebd-ea77dbaace58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:35.882904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 204/408] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:35.884248 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ce8eb-2477-48e8-aebd-ea77dbaace58 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:35.884703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:35.911981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e41aa-9dfc-4454-8971-178a700a90d2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:35.953817 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7a65-edef-4803-aeea-805697d60330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:35.955294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7a65-edef-4803-aeea-805697d60330 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:35.955783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7a65-edef-4803-aeea-805697d60330 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:35.956267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7a65-edef-4803-aeea-805697d60330 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1/groups/33f54fef6ed4400182390465adcd8a16/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:35.958981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7a65-edef-4803-aeea-805697d60330 None admin] RBAC: Authorizing `identity:create_grant(project_id=205237c47ce44b8b839b43198fb4c1f1, group_id=33f54fef6ed4400182390465adcd8a16, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:35.960808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7a65-edef-4803-aeea-805697d60330 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:35.978243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 205/409] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jun 20 10:51:35 2022] PUT /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1/groups/33f54fef6ed4400182390465adcd8a16/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:35.979609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7a65-edef-4803-aeea-805697d60330 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:36.056389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ce8eb-2477-48e8-aebd-ea77dbaace58 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:36.103992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:36.141042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f3d23-c29d-4fb7-83e8-f9dd5ddcfd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:36.142377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f3d23-c29d-4fb7-83e8-f9dd5ddcfd41 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:36.142857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f3d23-c29d-4fb7-83e8-f9dd5ddcfd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:36.143344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f3d23-c29d-4fb7-83e8-f9dd5ddcfd41 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1/groups/33f54fef6ed4400182390465adcd8a16/roles/ffab93457c0f4f64a29764f17759396e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:36.146241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f3d23-c29d-4fb7-83e8-f9dd5ddcfd41 None admin] RBAC: Authorizing `identity:create_grant(project_id=205237c47ce44b8b839b43198fb4c1f1, group_id=33f54fef6ed4400182390465adcd8a16, role_id=ffab93457c0f4f64a29764f17759396e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:36.149304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f3d23-c29d-4fb7-83e8-f9dd5ddcfd41 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:36.181163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 205/410] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jun 20 10:51:35 2022] PUT /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1/groups/33f54fef6ed4400182390465adcd8a16/roles/ffab93457c0f4f64a29764f17759396e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:36.206747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7a65-edef-4803-aeea-805697d60330 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:36.245025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc6972-f462-43b0-aa46-dd81675fcb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:36.245888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc6972-f462-43b0-aa46-dd81675fcb3d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:36.246587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc6972-f462-43b0-aa46-dd81675fcb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:36.246587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc6972-f462-43b0-aa46-dd81675fcb3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:36.257043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffc43c-d578-4810-be4e-310d2b8d857a None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:36.257678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffc43c-d578-4810-be4e-310d2b8d857a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:36.258192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffc43c-d578-4810-be4e-310d2b8d857a None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:36.258745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc6972-f462-43b0-aa46-dd81675fcb3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:36.260368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 206/411] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:36.265665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f118b57-f393-44c2-80cf-6e9b8154ac41 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:36.266290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f118b57-f393-44c2-80cf-6e9b8154ac41 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-ffcc6972-f462-43b0-aa46-dd81675fcb3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:36.266687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:36.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:36.268403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 206/412] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:51:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:36.269749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f118b57-f393-44c2-80cf-6e9b8154ac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:36.274767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc6972-f462-43b0-aa46-dd81675fcb3d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:36.289717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9f118b57-f393-44c2-80cf-6e9b8154ac41 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:36.291445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9f118b57-f393-44c2-80cf-6e9b8154ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:36.294750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:36.384732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 207/413] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:36.406282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc6972-f462-43b0-aa46-dd81675fcb3d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:36.409884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f118b57-f393-44c2-80cf-6e9b8154ac41 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:36.447816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35316704-b6ab-41f6-b3be-4620df2a1b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:36.448504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35316704-b6ab-41f6-b3be-4620df2a1b30 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:36.448834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35316704-b6ab-41f6-b3be-4620df2a1b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:36.449177 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35316704-b6ab-41f6-b3be-4620df2a1b30 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:36.449865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35316704-b6ab-41f6-b3be-4620df2a1b30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:36.450196 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35316704-b6ab-41f6-b3be-4620df2a1b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:36.452095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35316704-b6ab-41f6-b3be-4620df2a1b30 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:36.463717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 207/414] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:36.470329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35316704-b6ab-41f6-b3be-4620df2a1b30 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:36.525002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f118b57-f393-44c2-80cf-6e9b8154ac41 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:36.540171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:36.559123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:36.572258 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ca9ec-311e-4d2b-9c18-310a062d1f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:36.573395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ca9ec-311e-4d2b-9c18-310a062d1f6c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:36.573815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ca9ec-311e-4d2b-9c18-310a062d1f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:36.574118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ca9ec-311e-4d2b-9c18-310a062d1f6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:36.588887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35316704-b6ab-41f6-b3be-4620df2a1b30 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:36.593629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ca9ec-311e-4d2b-9c18-310a062d1f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-563ca9ec-311e-4d2b-9c18-310a062d1f6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:36.607935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:36.608885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:36.609699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:36.619215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 208/415] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:36.631515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ca9ec-311e-4d2b-9c18-310a062d1f6c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:36.670396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a081d-85cc-4984-b843-7903476f7c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:36.671195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a081d-85cc-4984-b843-7903476f7c61 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:36.671475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a081d-85cc-4984-b843-7903476f7c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:36.671750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a081d-85cc-4984-b843-7903476f7c61 None admin] PATH_INFO: `/v3/groups/69f8119ecb9443f0bd5e994f16671787` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:36.674514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a081d-85cc-4984-b843-7903476f7c61 None admin] RBAC: Authorizing `identity:get_group(group_id=69f8119ecb9443f0bd5e994f16671787)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:36.675995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a081d-85cc-4984-b843-7903476f7c61 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:36.681549 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 208/416] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:36 2022] GET /identity/v3/groups/69f8119ecb9443f0bd5e994f16671787 => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:36.684639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a081d-85cc-4984-b843-7903476f7c61 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:36.809205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a081d-85cc-4984-b843-7903476f7c61 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:36.859705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e609b-26d5-4bb5-a035-be197e6236a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:36.862949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e609b-26d5-4bb5-a035-be197e6236a3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:36.863431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e609b-26d5-4bb5-a035-be197e6236a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:36.865653 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ca9ec-311e-4d2b-9c18-310a062d1f6c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:36.866188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e609b-26d5-4bb5-a035-be197e6236a3 None admin] PATH_INFO: `/v3/projects/f76a7fac220343628fdd3080ad89317e` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:36.867605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e609b-26d5-4bb5-a035-be197e6236a3 None admin] RBAC: Authorizing `identity:get_project(project_id=f76a7fac220343628fdd3080ad89317e)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:36.870957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e609b-26d5-4bb5-a035-be197e6236a3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:36.879454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 209/417] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:36 2022] GET /identity/v3/projects/f76a7fac220343628fdd3080ad89317e => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:36.902602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e609b-26d5-4bb5-a035-be197e6236a3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:36.917986 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:36.933608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f9ea3-b889-4f26-8926-563402e7ab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:36.936538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f9ea3-b889-4f26-8926-563402e7ab14 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:36.936821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f9ea3-b889-4f26-8926-563402e7ab14 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:36.937097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f9ea3-b889-4f26-8926-563402e7ab14 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:36.937862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f9ea3-b889-4f26-8926-563402e7ab14 None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:36.938193 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f9ea3-b889-4f26-8926-563402e7ab14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:36.939652 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f9ea3-b889-4f26-8926-563402e7ab14 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:36.949582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 209/418] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:36.956010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f9ea3-b889-4f26-8926-563402e7ab14 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:37.016210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:37.074712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e609b-26d5-4bb5-a035-be197e6236a3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:37.113268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f9ea3-b889-4f26-8926-563402e7ab14 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:37.118033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bc022-fd99-4dc9-b29e-54a37c966a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:37.119067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bc022-fd99-4dc9-b29e-54a37c966a48 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:37.119451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bc022-fd99-4dc9-b29e-54a37c966a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:37.119842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bc022-fd99-4dc9-b29e-54a37c966a48 None admin] PATH_INFO: `/v3/projects/f76a7fac220343628fdd3080ad89317e/groups/69f8119ecb9443f0bd5e994f16671787/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:37.122825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bc022-fd99-4dc9-b29e-54a37c966a48 None admin] RBAC: Authorizing `identity:create_grant(project_id=f76a7fac220343628fdd3080ad89317e, group_id=69f8119ecb9443f0bd5e994f16671787, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:37.124904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bc022-fd99-4dc9-b29e-54a37c966a48 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:37.146156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 210/419] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Jun 20 10:51:36 2022] PUT /identity/v3/projects/f76a7fac220343628fdd3080ad89317e/groups/69f8119ecb9443f0bd5e994f16671787/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:37.148269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54ccd7-6b69-4da7-8da5-abdb78163a16 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:37.148767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54ccd7-6b69-4da7-8da5-abdb78163a16 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:37.149227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54ccd7-6b69-4da7-8da5-abdb78163a16 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:37.151903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 211/420] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:37.157706 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78395f-91d6-4d40-822b-0e3bbfb112bc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:37.157995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78395f-91d6-4d40-822b-0e3bbfb112bc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:37.158303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78395f-91d6-4d40-822b-0e3bbfb112bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:37.159386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780837d3-f3f0-43d4-9bee-f75509e95ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:37.160151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780837d3-f3f0-43d4-9bee-f75509e95ceb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:37.160616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780837d3-f3f0-43d4-9bee-f75509e95ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:37.160987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780837d3-f3f0-43d4-9bee-f75509e95ceb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:37.178266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cc78395f-91d6-4d40-822b-0e3bbfb112bc None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:37.179486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cc78395f-91d6-4d40-822b-0e3bbfb112bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:37.257039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:37.263252 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 212/421] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:37.290595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78395f-91d6-4d40-822b-0e3bbfb112bc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:37.326297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780837d3-f3f0-43d4-9bee-f75509e95ceb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:37.327785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780837d3-f3f0-43d4-9bee-f75509e95ceb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:37.409871 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78395f-91d6-4d40-822b-0e3bbfb112bc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:37.456172 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb233576-f269-4deb-9fb1-456a7a6b7bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:37.457036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb233576-f269-4deb-9fb1-456a7a6b7bad None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:37.457319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb233576-f269-4deb-9fb1-456a7a6b7bad None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:37.457594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb233576-f269-4deb-9fb1-456a7a6b7bad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:37.462461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:37.472623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb233576-f269-4deb-9fb1-456a7a6b7bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-fb233576-f269-4deb-9fb1-456a7a6b7bad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:37.483238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:37.484144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:37.484928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:37.485505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 213/422] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:37.491161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb233576-f269-4deb-9fb1-456a7a6b7bad None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:37.492968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 210/423] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:37.498302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780837d3-f3f0-43d4-9bee-f75509e95ceb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:37.567296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:37.612831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780837d3-f3f0-43d4-9bee-f75509e95ceb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:37.618759 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb233576-f269-4deb-9fb1-456a7a6b7bad None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:37.673611 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dd67d-48ce-4d47-a0da-347be4d83926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:37.674377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dd67d-48ce-4d47-a0da-347be4d83926 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:37.674657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dd67d-48ce-4d47-a0da-347be4d83926 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:37.674929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dd67d-48ce-4d47-a0da-347be4d83926 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:37.675676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dd67d-48ce-4d47-a0da-347be4d83926 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:37.675998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dd67d-48ce-4d47-a0da-347be4d83926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:37.677351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dd67d-48ce-4d47-a0da-347be4d83926 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:37.679240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:37.686637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246019ff-8d27-4cb3-b500-4403070f6de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:37.687784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246019ff-8d27-4cb3-b500-4403070f6de1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:37.688068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246019ff-8d27-4cb3-b500-4403070f6de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:37.688342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246019ff-8d27-4cb3-b500-4403070f6de1 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:37.692481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246019ff-8d27-4cb3-b500-4403070f6de1 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:37.694285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246019ff-8d27-4cb3-b500-4403070f6de1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:37.698725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 211/424] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:37 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:37.699696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 214/425] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:37.704422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246019ff-8d27-4cb3-b500-4403070f6de1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:37.707092 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dd67d-48ce-4d47-a0da-347be4d83926 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:37.807838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dd67d-48ce-4d47-a0da-347be4d83926 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:37.842073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cd18e-d644-4fd7-8921-30b6a744d762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:37.842734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cd18e-d644-4fd7-8921-30b6a744d762 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:37.843025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cd18e-d644-4fd7-8921-30b6a744d762 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:37.843348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cd18e-d644-4fd7-8921-30b6a744d762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:37.852938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:37.915594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cd18e-d644-4fd7-8921-30b6a744d762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:37.917097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cd18e-d644-4fd7-8921-30b6a744d762 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:37.946448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246019ff-8d27-4cb3-b500-4403070f6de1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:38.018343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 215/426] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:38.023039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b7699-4366-440b-bd76-f0bc6dcc36f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:38.023702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cd18e-d644-4fd7-8921-30b6a744d762 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:38.023922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b7699-4366-440b-bd76-f0bc6dcc36f1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.024207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b7699-4366-440b-bd76-f0bc6dcc36f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.024484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b7699-4366-440b-bd76-f0bc6dcc36f1 None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.030872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b7699-4366-440b-bd76-f0bc6dcc36f1 None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:38.032389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b7699-4366-440b-bd76-f0bc6dcc36f1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:38.034461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 212/427] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:37 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:38.040231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b7699-4366-440b-bd76-f0bc6dcc36f1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:38.058909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:38.142224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cd18e-d644-4fd7-8921-30b6a744d762 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:38.174435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:38.210015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3a294-4f2e-429e-bcab-4503fb36c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:38.210973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a294-4f2e-429e-bcab-4503fb36c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.211259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a294-4f2e-429e-bcab-4503fb36c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.211532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a294-4f2e-429e-bcab-4503fb36c7f4 None admin] PATH_INFO: `/v3/projects/9dd8fde445f5496fab61c82990f8ba09` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.213186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3a294-4f2e-429e-bcab-4503fb36c7f4 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8fde445f5496fab61c82990f8ba09)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:38.215381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3a294-4f2e-429e-bcab-4503fb36c7f4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:38.217471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 216/428] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:38 2022] GET /identity/v3/projects/9dd8fde445f5496fab61c82990f8ba09 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:38.222670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3a294-4f2e-429e-bcab-4503fb36c7f4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:38.230434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b7699-4366-440b-bd76-f0bc6dcc36f1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:38.314835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34e91d-ab52-48f7-9fae-acc5448dc109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:38.315530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34e91d-ab52-48f7-9fae-acc5448dc109 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.315894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34e91d-ab52-48f7-9fae-acc5448dc109 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.316095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34e91d-ab52-48f7-9fae-acc5448dc109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.316958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34e91d-ab52-48f7-9fae-acc5448dc109 None admin] RBAC: Adding query filter params (group.id=33f54fef6ed4400182390465adcd8a16, role.id=3e46d43e30ef494087da2c44f7ca702f, scope.project.id=9dd8fde445f5496fab61c82990f8ba09) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:38.317278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34e91d-ab52-48f7-9fae-acc5448dc109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:38.318797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34e91d-ab52-48f7-9fae-acc5448dc109 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:38.332557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 213/429] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jun 20 10:51:38 2022] GET /identity/v3/role_assignments?group.id=33f54fef6ed4400182390465adcd8a16&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 611 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:38.338545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:38.350810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3a294-4f2e-429e-bcab-4503fb36c7f4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:38.406526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11506754-28b9-4b1e-907e-63565b34dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:38.407336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11506754-28b9-4b1e-907e-63565b34dc8a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.407617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11506754-28b9-4b1e-907e-63565b34dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.407894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11506754-28b9-4b1e-907e-63565b34dc8a None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.409172 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11506754-28b9-4b1e-907e-63565b34dc8a None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:38.416635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11506754-28b9-4b1e-907e-63565b34dc8a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:38.419092 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 217/430] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:38 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:38.425335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34e91d-ab52-48f7-9fae-acc5448dc109 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:38.550253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34e91d-ab52-48f7-9fae-acc5448dc109 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:38.560800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501ad3f-b6c1-4794-a8db-e4a17f26d5d6 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.561100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501ad3f-b6c1-4794-a8db-e4a17f26d5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.561364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501ad3f-b6c1-4794-a8db-e4a17f26d5d6 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.563137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 218/431] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:38.568450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161de01-9471-4384-b70a-98a98cf6e455 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.568450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161de01-9471-4384-b70a-98a98cf6e455 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.568977 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161de01-9471-4384-b70a-98a98cf6e455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.582693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:38.588318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5161de01-9471-4384-b70a-98a98cf6e455 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:38.588542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca7edc-6bbd-4975-a9d2-82ddf1d7763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:38.589344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca7edc-6bbd-4975-a9d2-82ddf1d7763f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.590184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca7edc-6bbd-4975-a9d2-82ddf1d7763f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.590184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca7edc-6bbd-4975-a9d2-82ddf1d7763f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.590342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5161de01-9471-4384-b70a-98a98cf6e455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:38.590677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca7edc-6bbd-4975-a9d2-82ddf1d7763f None admin] RBAC: Adding query filter params (group.id=33f54fef6ed4400182390465adcd8a16, role.id=ffab93457c0f4f64a29764f17759396e, scope.project.id=9dd8fde445f5496fab61c82990f8ba09) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:38.590992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca7edc-6bbd-4975-a9d2-82ddf1d7763f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:38.592595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca7edc-6bbd-4975-a9d2-82ddf1d7763f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:38.599722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 214/432] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jun 20 10:51:38 2022] GET /identity/v3/role_assignments?group.id=33f54fef6ed4400182390465adcd8a16&scope.project.id=9dd8fde445f5496fab61c82990f8ba09&role.id=ffab93457c0f4f64a29764f17759396e => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:38.601075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2853d-f42e-4392-9989-d9b76d144c48 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.601367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2853d-f42e-4392-9989-d9b76d144c48 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.601648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2853d-f42e-4392-9989-d9b76d144c48 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.603315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 215/433] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:38.608815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7105a-348f-4bd0-93af-38cf2aab428e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.609227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7105a-348f-4bd0-93af-38cf2aab428e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.609923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7105a-348f-4bd0-93af-38cf2aab428e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.630054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d8a7105a-348f-4bd0-93af-38cf2aab428e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:38.631562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d8a7105a-348f-4bd0-93af-38cf2aab428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:38.687389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 219/434] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:38.712828 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161de01-9471-4384-b70a-98a98cf6e455 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:38.716264 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 216/435] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:38.754252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7105a-348f-4bd0-93af-38cf2aab428e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:38.828164 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:38.845148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161de01-9471-4384-b70a-98a98cf6e455 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:38.894178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7105a-348f-4bd0-93af-38cf2aab428e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:38.894769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcab8e7-650e-49de-ba2e-b5e7a8cd4b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:38.895752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcab8e7-650e-49de-ba2e-b5e7a8cd4b3c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.896239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcab8e7-650e-49de-ba2e-b5e7a8cd4b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.896513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcab8e7-650e-49de-ba2e-b5e7a8cd4b3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.913670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcab8e7-650e-49de-ba2e-b5e7a8cd4b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1dcab8e7-650e-49de-ba2e-b5e7a8cd4b3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:38.924598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:38.925538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:38.926345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:38.926946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 220/436] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:38.932678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcab8e7-650e-49de-ba2e-b5e7a8cd4b3c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:38.934529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:38.948214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b074c0-52cb-4634-ae5f-c9b6ac376e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:38.949795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b074c0-52cb-4634-ae5f-c9b6ac376e16 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:38.950673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b074c0-52cb-4634-ae5f-c9b6ac376e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:38.951150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b074c0-52cb-4634-ae5f-c9b6ac376e16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:38.968123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b074c0-52cb-4634-ae5f-c9b6ac376e16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-74b074c0-52cb-4634-ae5f-c9b6ac376e16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:38.976894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:38.977758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 10:51:38.978618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:38.979723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 217/437] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:38.985574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b074c0-52cb-4634-ae5f-c9b6ac376e16 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:39.069751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcab8e7-650e-49de-ba2e-b5e7a8cd4b3c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:39.109976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b074c0-52cb-4634-ae5f-c9b6ac376e16 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:39.114125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03e583-77e9-4323-bc72-6d2f45edb5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:39.115053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e583-77e9-4323-bc72-6d2f45edb5cf None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.115615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e583-77e9-4323-bc72-6d2f45edb5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.116082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e583-77e9-4323-bc72-6d2f45edb5cf None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.117026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03e583-77e9-4323-bc72-6d2f45edb5cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:39.117525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03e583-77e9-4323-bc72-6d2f45edb5cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:39.119675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03e583-77e9-4323-bc72-6d2f45edb5cf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:39.129715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 221/438] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:39.135112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03e583-77e9-4323-bc72-6d2f45edb5cf None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:39.149333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:39.155331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba39849-4e79-4456-8953-320667f515e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:39.156095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39849-4e79-4456-8953-320667f515e4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.156382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39849-4e79-4456-8953-320667f515e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.156656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39849-4e79-4456-8953-320667f515e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.157410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba39849-4e79-4456-8953-320667f515e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:39.157717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba39849-4e79-4456-8953-320667f515e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:39.159316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba39849-4e79-4456-8953-320667f515e4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:39.168271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 218/439] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:51:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:39.174015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba39849-4e79-4456-8953-320667f515e4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:39.232947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:39.276786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03e583-77e9-4323-bc72-6d2f45edb5cf None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:39.300877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba39849-4e79-4456-8953-320667f515e4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:39.313185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acda88-6975-433b-b171-1e51d169949a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:39.313843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acda88-6975-433b-b171-1e51d169949a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.314151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acda88-6975-433b-b171-1e51d169949a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.314425 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acda88-6975-433b-b171-1e51d169949a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.342021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5463a-6479-4f45-a048-726ab695d892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:39.343101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5463a-6479-4f45-a048-726ab695d892 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.343524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5463a-6479-4f45-a048-726ab695d892 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.343926 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5463a-6479-4f45-a048-726ab695d892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.388474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acda88-6975-433b-b171-1e51d169949a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:39.389615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acda88-6975-433b-b171-1e51d169949a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:39.401314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:39.420747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5463a-6479-4f45-a048-726ab695d892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:39.422308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5463a-6479-4f45-a048-726ab695d892 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:39.475027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:39.498583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 222/440] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:39.499596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acda88-6975-433b-b171-1e51d169949a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:39.534127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 219/441] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:39.540048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5463a-6479-4f45-a048-726ab695d892 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:39.604090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acda88-6975-433b-b171-1e51d169949a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:39.615410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:39.639089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b93a-510b-46e0-a67b-940f8978afbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:39.640077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b93a-510b-46e0-a67b-940f8978afbf None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.640426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b93a-510b-46e0-a67b-940f8978afbf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.640766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b93a-510b-46e0-a67b-940f8978afbf None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.642007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68b93a-510b-46e0-a67b-940f8978afbf None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:39.643494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68b93a-510b-46e0-a67b-940f8978afbf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:39.644886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5463a-6479-4f45-a048-726ab695d892 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:39.645214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 223/442] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:39 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:39.650464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b93a-510b-46e0-a67b-940f8978afbf None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:39.685220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521ff6c-87d9-4616-b567-04b00ad841c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:39.686452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521ff6c-87d9-4616-b567-04b00ad841c2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.686865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521ff6c-87d9-4616-b567-04b00ad841c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.687273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521ff6c-87d9-4616-b567-04b00ad841c2 None admin] PATH_INFO: `/v3/projects/205237c47ce44b8b839b43198fb4c1f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.688609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521ff6c-87d9-4616-b567-04b00ad841c2 None admin] RBAC: Authorizing `identity:get_project(project_id=205237c47ce44b8b839b43198fb4c1f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:39.690232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521ff6c-87d9-4616-b567-04b00ad841c2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:39.692542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 220/443] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:39 2022] GET /identity/v3/projects/205237c47ce44b8b839b43198fb4c1f1 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:39.697485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521ff6c-87d9-4616-b567-04b00ad841c2 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:39.717079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:39.757725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b93a-510b-46e0-a67b-940f8978afbf None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:39.796008 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3274f1-63ef-47b6-8fee-09c2cc78df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:39.796884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3274f1-63ef-47b6-8fee-09c2cc78df7d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.797175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3274f1-63ef-47b6-8fee-09c2cc78df7d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.797436 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3274f1-63ef-47b6-8fee-09c2cc78df7d None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.798847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3274f1-63ef-47b6-8fee-09c2cc78df7d None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:39.800569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3274f1-63ef-47b6-8fee-09c2cc78df7d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:39.801527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521ff6c-87d9-4616-b567-04b00ad841c2 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:39.802213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 224/444] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:39 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:39.803479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8898f0b-5268-4bd5-b24f-38cad60559e8 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.803855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8898f0b-5268-4bd5-b24f-38cad60559e8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.804128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8898f0b-5268-4bd5-b24f-38cad60559e8 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.806119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 225/445] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:39.808035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8898f0b-5268-4bd5-b24f-38cad60559e8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:39.837273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaed20-414b-4d1a-ab5a-7bb91e2c2b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:39.838165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaed20-414b-4d1a-ab5a-7bb91e2c2b78 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.838443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaed20-414b-4d1a-ab5a-7bb91e2c2b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.838714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaed20-414b-4d1a-ab5a-7bb91e2c2b78 None admin] PATH_INFO: `/v3/groups/33f54fef6ed4400182390465adcd8a16` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.839928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beaed20-414b-4d1a-ab5a-7bb91e2c2b78 None admin] RBAC: Authorizing `identity:get_group(group_id=33f54fef6ed4400182390465adcd8a16)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:39.841391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beaed20-414b-4d1a-ab5a-7bb91e2c2b78 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:39.846132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 221/446] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:39 2022] GET /identity/v3/groups/33f54fef6ed4400182390465adcd8a16 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:39.846132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cbe9b-89c3-4a49-a494-196410e5b08c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.846132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cbe9b-89c3-4a49-a494-196410e5b08c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.846132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cbe9b-89c3-4a49-a494-196410e5b08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.862663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-670cbe9b-89c3-4a49-a494-196410e5b08c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:39.863863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-670cbe9b-89c3-4a49-a494-196410e5b08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:39.865808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:39.937379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 222/447] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:39.937580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8898f0b-5268-4bd5-b24f-38cad60559e8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:39.938300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cbe9b-89c3-4a49-a494-196410e5b08c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:39.957289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:39.979997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accdec7-e6db-43c3-8fb4-1e33f8a2cb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:39.980522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accdec7-e6db-43c3-8fb4-1e33f8a2cb93 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:39.980806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accdec7-e6db-43c3-8fb4-1e33f8a2cb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:39.981081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6accdec7-e6db-43c3-8fb4-1e33f8a2cb93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:39.981906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accdec7-e6db-43c3-8fb4-1e33f8a2cb93 None admin] RBAC: Adding query filter params (group.id=33f54fef6ed4400182390465adcd8a16, role.id=ffab93457c0f4f64a29764f17759396e, scope.project.id=205237c47ce44b8b839b43198fb4c1f1) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:39.982247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accdec7-e6db-43c3-8fb4-1e33f8a2cb93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:39.984090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6accdec7-e6db-43c3-8fb4-1e33f8a2cb93 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:39.990809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 226/448] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jun 20 10:51:39 2022] GET /identity/v3/role_assignments?group.id=33f54fef6ed4400182390465adcd8a16&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=ffab93457c0f4f64a29764f17759396e => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:39.991581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accdec7-e6db-43c3-8fb4-1e33f8a2cb93 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:40.061954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cbe9b-89c3-4a49-a494-196410e5b08c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:40.103966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3f2f9-33e2-41f3-8e6d-070386a896b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:40.104646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3f2f9-33e2-41f3-8e6d-070386a896b6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:40.104964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3f2f9-33e2-41f3-8e6d-070386a896b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:40.105285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3f2f9-33e2-41f3-8e6d-070386a896b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:40.106325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3f2f9-33e2-41f3-8e6d-070386a896b6 None admin] RBAC: Adding query filter params (group.id=33f54fef6ed4400182390465adcd8a16, role.id=3e46d43e30ef494087da2c44f7ca702f, scope.project.id=205237c47ce44b8b839b43198fb4c1f1) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:40.106649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3f2f9-33e2-41f3-8e6d-070386a896b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:40.108172 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3f2f9-33e2-41f3-8e6d-070386a896b6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:40.112802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6accdec7-e6db-43c3-8fb4-1e33f8a2cb93 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:40.122393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 223/449] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jun 20 10:51:39 2022] GET /identity/v3/role_assignments?group.id=33f54fef6ed4400182390465adcd8a16&scope.project.id=205237c47ce44b8b839b43198fb4c1f1&role.id=3e46d43e30ef494087da2c44f7ca702f => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:40.133138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:40.159956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff50b3-7fe5-4d68-b4ac-3d445726e4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:40.160791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff50b3-7fe5-4d68-b4ac-3d445726e4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:40.161076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff50b3-7fe5-4d68-b4ac-3d445726e4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:40.161356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff50b3-7fe5-4d68-b4ac-3d445726e4d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:40.174151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff50b3-7fe5-4d68-b4ac-3d445726e4d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6bff50b3-7fe5-4d68-b4ac-3d445726e4d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:40.182056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:40.183250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:40.184034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 227/450] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:51:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:40.191155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3f2f9-33e2-41f3-8e6d-070386a896b6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:40.284088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:40.316386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3f2f9-33e2-41f3-8e6d-070386a896b6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:40.351561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba7f78-1596-46a7-b11e-14cdc2c11f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:40.352239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba7f78-1596-46a7-b11e-14cdc2c11f87 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:40.352528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba7f78-1596-46a7-b11e-14cdc2c11f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:40.352802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba7f78-1596-46a7-b11e-14cdc2c11f87 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:40.353536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba7f78-1596-46a7-b11e-14cdc2c11f87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:40.353844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba7f78-1596-46a7-b11e-14cdc2c11f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:40.355314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba7f78-1596-46a7-b11e-14cdc2c11f87 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:40.363917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 224/451] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:40.369930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff50b3-7fe5-4d68-b4ac-3d445726e4d1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:40.479446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff50b3-7fe5-4d68-b4ac-3d445726e4d1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:40.513320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c06c02-a270-47d7-859d-e1f726e05055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:40.513976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c06c02-a270-47d7-859d-e1f726e05055 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:40.514296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c06c02-a270-47d7-859d-e1f726e05055 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:40.514571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c06c02-a270-47d7-859d-e1f726e05055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:40.554233 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:40.583979 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c06c02-a270-47d7-859d-e1f726e05055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:40.585383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c06c02-a270-47d7-859d-e1f726e05055 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:40.681579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 228/452] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:40.688237 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba7f78-1596-46a7-b11e-14cdc2c11f87 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:40.794230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba7f78-1596-46a7-b11e-14cdc2c11f87 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:40.829566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e554128-56f1-44e3-8309-a4396edee0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:40.830450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e554128-56f1-44e3-8309-a4396edee0fa None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:40.830728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e554128-56f1-44e3-8309-a4396edee0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:40.831004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e554128-56f1-44e3-8309-a4396edee0fa None admin] PATH_INFO: `/v3/projects/f76a7fac220343628fdd3080ad89317e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:40.832319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e554128-56f1-44e3-8309-a4396edee0fa None admin] RBAC: Authorizing `identity:get_project(project_id=f76a7fac220343628fdd3080ad89317e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:40.834236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e554128-56f1-44e3-8309-a4396edee0fa None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:40.835865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 225/453] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:51:40 2022] GET /identity/v3/projects/f76a7fac220343628fdd3080ad89317e => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:40.842768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c06c02-a270-47d7-859d-e1f726e05055 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:40.932406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:40.970026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c06c02-a270-47d7-859d-e1f726e05055 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:41.008349 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a67acd-7139-49b1-911c-f6e6f78f13d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:41.009199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a67acd-7139-49b1-911c-f6e6f78f13d8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:41.009479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a67acd-7139-49b1-911c-f6e6f78f13d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:41.009743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a67acd-7139-49b1-911c-f6e6f78f13d8 None admin] PATH_INFO: `/v3/groups/69f8119ecb9443f0bd5e994f16671787` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:41.011055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a67acd-7139-49b1-911c-f6e6f78f13d8 None admin] RBAC: Authorizing `identity:get_group(group_id=69f8119ecb9443f0bd5e994f16671787)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:41.013004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a67acd-7139-49b1-911c-f6e6f78f13d8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:41.014647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 229/454] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Jun 20 10:51:40 2022] GET /identity/v3/groups/69f8119ecb9443f0bd5e994f16671787 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:41.020817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e554128-56f1-44e3-8309-a4396edee0fa None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:41.033539 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:41.126935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e554128-56f1-44e3-8309-a4396edee0fa None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:41.164646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41cbfa4-28e1-47c5-8c0b-9a9cb8502c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:41.165289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cbfa4-28e1-47c5-8c0b-9a9cb8502c2e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:41.165567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cbfa4-28e1-47c5-8c0b-9a9cb8502c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:41.165849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41cbfa4-28e1-47c5-8c0b-9a9cb8502c2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:41.166718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cbfa4-28e1-47c5-8c0b-9a9cb8502c2e None admin] RBAC: Adding query filter params (group.id=69f8119ecb9443f0bd5e994f16671787, role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=f76a7fac220343628fdd3080ad89317e) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:41.167031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cbfa4-28e1-47c5-8c0b-9a9cb8502c2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:41.168526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41cbfa4-28e1-47c5-8c0b-9a9cb8502c2e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:41.175222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 226/455] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Jun 20 10:51:41 2022] GET /identity/v3/role_assignments?group.id=69f8119ecb9443f0bd5e994f16671787&scope.project.id=f76a7fac220343628fdd3080ad89317e&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:43.649999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af6b83-a450-4297-acf7-20baf906ee08 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:43.650548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af6b83-a450-4297-acf7-20baf906ee08 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:43.650548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af6b83-a450-4297-acf7-20baf906ee08 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:43.652189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 230/456] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:43.658177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3f256-df9b-4b69-a0ba-aa8700563717 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:43.658657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3f256-df9b-4b69-a0ba-aa8700563717 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:43.658734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3f256-df9b-4b69-a0ba-aa8700563717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:43.683106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-66e3f256-df9b-4b69-a0ba-aa8700563717 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:43.684607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-66e3f256-df9b-4b69-a0ba-aa8700563717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:43.770762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 227/457] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:43.815682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af6b83-a450-4297-acf7-20baf906ee08 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:43.969822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af6b83-a450-4297-acf7-20baf906ee08 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:44.016798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439cb10-630f-40d2-93e3-2f2b0a9a22dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:44.017614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439cb10-630f-40d2-93e3-2f2b0a9a22dd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.017895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439cb10-630f-40d2-93e3-2f2b0a9a22dd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.018195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439cb10-630f-40d2-93e3-2f2b0a9a22dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.031034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868faf1-b383-4442-a7be-c52bcefe284a None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.031326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868faf1-b383-4442-a7be-c52bcefe284a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.031603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868faf1-b383-4442-a7be-c52bcefe284a None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.031837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:44.033569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 228/458] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:44.040800 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee7664-101e-4912-9165-289ac327a1b6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.041071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439cb10-630f-40d2-93e3-2f2b0a9a22dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:44.041170 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee7664-101e-4912-9165-289ac327a1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.041322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee7664-101e-4912-9165-289ac327a1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.042527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439cb10-630f-40d2-93e3-2f2b0a9a22dd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:44.050415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:44.075047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-04ee7664-101e-4912-9165-289ac327a1b6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-0439cb10-630f-40d2-93e3-2f2b0a9a22dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:44.076152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:44.077184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:44.078402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:44.078402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:44.078402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:44.078402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:44.078402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 231/459] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:44.078906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba80a4-88ef-4ff6-b9b0-c474a9eac7d5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.079187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba80a4-88ef-4ff6-b9b0-c474a9eac7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.079381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-04ee7664-101e-4912-9165-289ac327a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:44.079480 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba80a4-88ef-4ff6-b9b0-c474a9eac7d5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.080909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 232/460] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:44.084103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba80a4-88ef-4ff6-b9b0-c474a9eac7d5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:44.162272 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 229/461] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:44.163538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50d371-f4be-4318-a416-9516d60fb43b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.163823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50d371-f4be-4318-a416-9516d60fb43b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.164097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50d371-f4be-4318-a416-9516d60fb43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.194812 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-cb50d371-f4be-4318-a416-9516d60fb43b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:44.196278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cb50d371-f4be-4318-a416-9516d60fb43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:44.219332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba80a4-88ef-4ff6-b9b0-c474a9eac7d5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:44.255850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef896023-cfd4-4661-b2e2-568c059c020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:44.256808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896023-cfd4-4661-b2e2-568c059c020c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.257278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896023-cfd4-4661-b2e2-568c059c020c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.257711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef896023-cfd4-4661-b2e2-568c059c020c None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.258640 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef896023-cfd4-4661-b2e2-568c059c020c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:44.259090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef896023-cfd4-4661-b2e2-568c059c020c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:44.260628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef896023-cfd4-4661-b2e2-568c059c020c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:44.267399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 230/462] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:44.267399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50d371-f4be-4318-a416-9516d60fb43b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:44.275940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 233/463] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:44.277732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f865c-b5eb-48fd-9f87-867967feb583 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.278296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f865c-b5eb-48fd-9f87-867967feb583 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.278716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f865c-b5eb-48fd-9f87-867967feb583 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.280458 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 234/464] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:44.282205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f865c-b5eb-48fd-9f87-867967feb583 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:44.335356 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:44.343369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:44.385568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50d371-f4be-4318-a416-9516d60fb43b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:44.405700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f865c-b5eb-48fd-9f87-867967feb583 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:44.421157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362f33b-22a0-431e-b17a-1d2d40229a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:44.421892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362f33b-22a0-431e-b17a-1d2d40229a8b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.422196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362f33b-22a0-431e-b17a-1d2d40229a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.422465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362f33b-22a0-431e-b17a-1d2d40229a8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.432642 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362f33b-22a0-431e-b17a-1d2d40229a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:44.434315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362f33b-22a0-431e-b17a-1d2d40229a8b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:44.446210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2c2d6-1cbf-45c7-96df-069a374ffda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:44.447055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2c2d6-1cbf-45c7-96df-069a374ffda8 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.447475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2c2d6-1cbf-45c7-96df-069a374ffda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.447989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2c2d6-1cbf-45c7-96df-069a374ffda8 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.448826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2c2d6-1cbf-45c7-96df-069a374ffda8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:44.450571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2c2d6-1cbf-45c7-96df-069a374ffda8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-9362f33b-22a0-431e-b17a-1d2d40229a8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:44.451659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:44.452656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:44.453534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:44.453534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:44.453534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:44.453534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:44.453534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 231/465] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:44.454013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159237c-8a42-4666-883e-a051840c1af1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.454316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159237c-8a42-4666-883e-a051840c1af1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.454588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159237c-8a42-4666-883e-a051840c1af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.473566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f159237c-8a42-4666-883e-a051840c1af1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:44.474729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f159237c-8a42-4666-883e-a051840c1af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:44.486631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 235/466] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:51:44 2022] POST /identity/v3/users => generated 246 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:44.487628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2c2d6-1cbf-45c7-96df-069a374ffda8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:44.580873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 232/467] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:44.587625 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159237c-8a42-4666-883e-a051840c1af1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:44.642887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2c2d6-1cbf-45c7-96df-069a374ffda8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:44.652645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:44.662582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:44.695612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc7d53-cb70-451a-9cf4-4f0aafd80585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:44.696768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7d53-cb70-451a-9cf4-4f0aafd80585 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.697278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7d53-cb70-451a-9cf4-4f0aafd80585 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.697741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc7d53-cb70-451a-9cf4-4f0aafd80585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.711877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc7d53-cb70-451a-9cf4-4f0aafd80585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:44.713632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc7d53-cb70-451a-9cf4-4f0aafd80585 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-75cc7d53-cb70-451a-9cf4-4f0aafd80585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:44.734362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:44.735250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:44.736061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:44.736061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:44.736061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:44.736061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:44.741570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 236/468] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:44.741570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc7d53-cb70-451a-9cf4-4f0aafd80585 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:44.741778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159237c-8a42-4666-883e-a051840c1af1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:44.788462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8131703-dbb8-4f57-b151-2e04b461d6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:44.789532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131703-dbb8-4f57-b151-2e04b461d6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.790459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131703-dbb8-4f57-b151-2e04b461d6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.790960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131703-dbb8-4f57-b151-2e04b461d6e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.792433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8131703-dbb8-4f57-b151-2e04b461d6e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:44.793284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8131703-dbb8-4f57-b151-2e04b461d6e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:44.796334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8131703-dbb8-4f57-b151-2e04b461d6e0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:44.817733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 233/469] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:44.817733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8131703-dbb8-4f57-b151-2e04b461d6e0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:44.875984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc7d53-cb70-451a-9cf4-4f0aafd80585 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:44.929621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ade8e-a07c-4b9b-a29e-ae646fc29902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:44.930500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ade8e-a07c-4b9b-a29e-ae646fc29902 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.930787 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ade8e-a07c-4b9b-a29e-ae646fc29902 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.931062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ade8e-a07c-4b9b-a29e-ae646fc29902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.943033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ade8e-a07c-4b9b-a29e-ae646fc29902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:44.944038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8131703-dbb8-4f57-b151-2e04b461d6e0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:44.944683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ade8e-a07c-4b9b-a29e-ae646fc29902 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:44.947190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2c8ade8e-a07c-4b9b-a29e-ae646fc29902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:44.963103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:44.963995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:44.965545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:44.965545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:44.965545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:44.965545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:44.965545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 237/470] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:44.966829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ade8e-a07c-4b9b-a29e-ae646fc29902 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:44.981229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:44.992862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37169360-6e02-4c62-8fec-480edb8c7375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:44.993702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37169360-6e02-4c62-8fec-480edb8c7375 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:44.994414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37169360-6e02-4c62-8fec-480edb8c7375 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:44.994636 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37169360-6e02-4c62-8fec-480edb8c7375 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:44.995473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37169360-6e02-4c62-8fec-480edb8c7375 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:44.995954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37169360-6e02-4c62-8fec-480edb8c7375 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:44.997554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37169360-6e02-4c62-8fec-480edb8c7375 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:45.019829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 234/471] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:45.019829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37169360-6e02-4c62-8fec-480edb8c7375 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:45.090184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ade8e-a07c-4b9b-a29e-ae646fc29902 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:45.126300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d6345-f713-47a8-bcd6-e20dc39a1b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:45.127133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d6345-f713-47a8-bcd6-e20dc39a1b80 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:45.127516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d6345-f713-47a8-bcd6-e20dc39a1b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:45.128027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d6345-f713-47a8-bcd6-e20dc39a1b80 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:45.128772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d6345-f713-47a8-bcd6-e20dc39a1b80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:45.130417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d6345-f713-47a8-bcd6-e20dc39a1b80 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:45.131141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37169360-6e02-4c62-8fec-480edb8c7375 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:45.153496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 238/472] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:51:44 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:45.154495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d6345-f713-47a8-bcd6-e20dc39a1b80 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:45.171900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc94e1-e3c4-4a1e-a24e-f45668099046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:45.172959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc94e1-e3c4-4a1e-a24e-f45668099046 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:45.173376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc94e1-e3c4-4a1e-a24e-f45668099046 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:45.173780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc94e1-e3c4-4a1e-a24e-f45668099046 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:45.174862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc94e1-e3c4-4a1e-a24e-f45668099046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:45.175316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc94e1-e3c4-4a1e-a24e-f45668099046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:45.177143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc94e1-e3c4-4a1e-a24e-f45668099046 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:45.197075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 235/473] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:45.201023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aad95-bc4d-42b1-8054-6e87ecaf9064 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:45.201986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aad95-bc4d-42b1-8054-6e87ecaf9064 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:45.202504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aad95-bc4d-42b1-8054-6e87ecaf9064 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:45.204178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 236/474] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:45.205157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511aad95-bc4d-42b1-8054-6e87ecaf9064 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:45.236820 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:45.243886 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:45.288232 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d6345-f713-47a8-bcd6-e20dc39a1b80 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:45.329792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706d99f-b60c-4ac5-8cf7-68558fa47ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:45.331230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706d99f-b60c-4ac5-8cf7-68558fa47ca8 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:45.331959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706d99f-b60c-4ac5-8cf7-68558fa47ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:45.332374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706d99f-b60c-4ac5-8cf7-68558fa47ca8 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:45.334467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706d99f-b60c-4ac5-8cf7-68558fa47ca8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:45.336678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706d99f-b60c-4ac5-8cf7-68558fa47ca8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:45.346250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511aad95-bc4d-42b1-8054-6e87ecaf9064 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:45.363890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 239/475] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:51:45 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:45.369046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc7f19-7447-4b7d-92e3-db2f1095f850 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:45.369046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc7f19-7447-4b7d-92e3-db2f1095f850 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:45.369046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc7f19-7447-4b7d-92e3-db2f1095f850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:45.389561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cedc7f19-7447-4b7d-92e3-db2f1095f850 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:45.390822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cedc7f19-7447-4b7d-92e3-db2f1095f850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:45.399758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8303318-f889-47e5-b65c-58db90a0c670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:45.400573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8303318-f889-47e5-b65c-58db90a0c670 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:45.401573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8303318-f889-47e5-b65c-58db90a0c670 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:45.401861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8303318-f889-47e5-b65c-58db90a0c670 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:45.403347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8303318-f889-47e5-b65c-58db90a0c670 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:45.406632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8303318-f889-47e5-b65c-58db90a0c670 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:45.437223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 237/476] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:51:45 2022] POST /identity/v3/users => generated 249 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:45.484815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 240/477] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:45.510973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8303318-f889-47e5-b65c-58db90a0c670 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:45.616776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:45.642199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8303318-f889-47e5-b65c-58db90a0c670 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:45.682569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e27d-241a-41d0-9b15-f104cc93604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:45.684039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e27d-241a-41d0-9b15-f104cc93604a None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:45.684313 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e27d-241a-41d0-9b15-f104cc93604a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:45.684591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e27d-241a-41d0-9b15-f104cc93604a None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:45.685342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8e27d-241a-41d0-9b15-f104cc93604a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:45.686800 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8e27d-241a-41d0-9b15-f104cc93604a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:45.695916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.sql.core [None req-37c8e27d-241a-41d0-9b15-f104cc93604a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 20 10:51:45.695916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 20 10:51:45.695916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [parameters: {'id': 'da60152be2e44265afb09678da0f71f8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 20 10:51:45.695916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80132) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-37c8e27d-241a-41d0-9b15-f104cc93604a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application conn.query(q) Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application result.read() Jun 20 10:51:45.713473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return ref.to_dict() Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 10:51:45.714517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application next(self.gen) Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application yield resource Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application next(self.gen) Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self.session.rollback() Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self.force_reraise() Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise self.value Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application session.commit() Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 20 10:51:45.715472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._prepare_impl() Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self.session.flush() Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._flush(objects) Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application compat.raise_( Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application flush_context.execute() Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rec.execute(self) Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 20 10:51:45.717118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 10:51:45.719141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application conn.query(q) Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application result.read() Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 10:51:45.721079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [parameters: {'id': 'da60152be2e44265afb09678da0f71f8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:45.723038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 20 10:51:45.725837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:45.727829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 238/478] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jun 20 10:51:45 2022] POST /identity/v3/roles => generated 144 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:45.727959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc7f19-7447-4b7d-92e3-db2f1095f850 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:45.782510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:45.838009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc7f19-7447-4b7d-92e3-db2f1095f850 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:45.874899 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f136d38-f75a-4c1a-8b92-039e97e2be4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:45.875720 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f136d38-f75a-4c1a-8b92-039e97e2be4d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:45.876001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f136d38-f75a-4c1a-8b92-039e97e2be4d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:45.876274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f136d38-f75a-4c1a-8b92-039e97e2be4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:45.886675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f136d38-f75a-4c1a-8b92-039e97e2be4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-5f136d38-f75a-4c1a-8b92-039e97e2be4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:45.894758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:45.896802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:45.897876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 241/479] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:51:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:45.901632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e27d-241a-41d0-9b15-f104cc93604a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:46.051840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e27d-241a-41d0-9b15-f104cc93604a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:46.099047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020fb1d-f35d-42f0-a504-2a1d9bc0d05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:46.099734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020fb1d-f35d-42f0-a504-2a1d9bc0d05c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:46.100021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020fb1d-f35d-42f0-a504-2a1d9bc0d05c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:46.100300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020fb1d-f35d-42f0-a504-2a1d9bc0d05c None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:46.101061 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020fb1d-f35d-42f0-a504-2a1d9bc0d05c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:46.101371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020fb1d-f35d-42f0-a504-2a1d9bc0d05c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:46.102809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020fb1d-f35d-42f0-a504-2a1d9bc0d05c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:46.111321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 239/480] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:46.835120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e211517-23fb-44db-a0cd-8087944f6cb6 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:46.835636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e211517-23fb-44db-a0cd-8087944f6cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:46.835636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e211517-23fb-44db-a0cd-8087944f6cb6 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:46.837310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 242/481] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:46.842965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db123f-bae9-48fb-bebd-0b6b9c2c2df3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:46.843422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db123f-bae9-48fb-bebd-0b6b9c2c2df3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:46.843599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db123f-bae9-48fb-bebd-0b6b9c2c2df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:46.859589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:46.867214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-37db123f-bae9-48fb-bebd-0b6b9c2c2df3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:46.868426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-37db123f-bae9-48fb-bebd-0b6b9c2c2df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:46.964601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 240/482] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:47.002382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e211517-23fb-44db-a0cd-8087944f6cb6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:47.109769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e211517-23fb-44db-a0cd-8087944f6cb6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:47.123001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:47.144451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc5582-ca71-4f4a-a686-7ffc9d4d8cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:47.145258 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc5582-ca71-4f4a-a686-7ffc9d4d8cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:47.145542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc5582-ca71-4f4a-a686-7ffc9d4d8cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:47.145819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc5582-ca71-4f4a-a686-7ffc9d4d8cc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:47.156598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc5582-ca71-4f4a-a686-7ffc9d4d8cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-54fc5582-ca71-4f4a-a686-7ffc9d4d8cc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:47.164242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:47.165202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:47.166028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 243/483] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:47.173088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db123f-bae9-48fb-bebd-0b6b9c2c2df3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:47.296865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db123f-bae9-48fb-bebd-0b6b9c2c2df3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:47.347842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258182b4-c97d-4648-b962-d121585ba4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:47.348521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258182b4-c97d-4648-b962-d121585ba4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:47.348804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258182b4-c97d-4648-b962-d121585ba4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:47.349079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258182b4-c97d-4648-b962-d121585ba4e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:47.349837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258182b4-c97d-4648-b962-d121585ba4e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:47.350172 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258182b4-c97d-4648-b962-d121585ba4e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:47.351579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258182b4-c97d-4648-b962-d121585ba4e1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:47.354054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:47.362799 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 241/484] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:47.371514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc5582-ca71-4f4a-a686-7ffc9d4d8cc9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:47.475459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc5582-ca71-4f4a-a686-7ffc9d4d8cc9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:47.521523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69e9c3-b6a4-4f18-a054-d4844273790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:47.522375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69e9c3-b6a4-4f18-a054-d4844273790d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:47.522657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69e9c3-b6a4-4f18-a054-d4844273790d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:47.522937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69e9c3-b6a4-4f18-a054-d4844273790d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:47.537977 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69e9c3-b6a4-4f18-a054-d4844273790d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:47.539484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69e9c3-b6a4-4f18-a054-d4844273790d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2d69e9c3-b6a4-4f18-a054-d4844273790d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:47.566898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:47.568953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:47.571050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:47.571050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:47.571050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:47.571050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:47.571050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 244/485] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:47.577342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258182b4-c97d-4648-b962-d121585ba4e1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:47.716780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258182b4-c97d-4648-b962-d121585ba4e1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:47.760708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7764c3-19f6-4a2c-a969-98677e561e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:47.761466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7764c3-19f6-4a2c-a969-98677e561e8a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:47.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7764c3-19f6-4a2c-a969-98677e561e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:47.762308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7764c3-19f6-4a2c-a969-98677e561e8a None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:47.763042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7764c3-19f6-4a2c-a969-98677e561e8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:47.763354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7764c3-19f6-4a2c-a969-98677e561e8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:47.764746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7764c3-19f6-4a2c-a969-98677e561e8a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:47.781360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 242/486] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:47.787106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69e9c3-b6a4-4f18-a054-d4844273790d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:47.823097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:47.919268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69e9c3-b6a4-4f18-a054-d4844273790d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:47.953997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d453e6-d984-428c-bfff-5a9cd224d9ab None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:47.954522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d453e6-d984-428c-bfff-5a9cd224d9ab None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:47.954633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d453e6-d984-428c-bfff-5a9cd224d9ab None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:47.955955 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 243/487] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:47.961533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5488816-5536-44fa-876b-897b6ebc4621 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:47.961861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5488816-5536-44fa-876b-897b6ebc4621 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:47.962114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5488816-5536-44fa-876b-897b6ebc4621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:47.963599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee982a7-d0f7-4187-b3cc-9d5a51f7c30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:47.964465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee982a7-d0f7-4187-b3cc-9d5a51f7c30e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:47.964785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee982a7-d0f7-4187-b3cc-9d5a51f7c30e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:47.965065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee982a7-d0f7-4187-b3cc-9d5a51f7c30e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:47.974196 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee982a7-d0f7-4187-b3cc-9d5a51f7c30e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:47.975355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee982a7-d0f7-4187-b3cc-9d5a51f7c30e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:47.985627 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f5488816-5536-44fa-876b-897b6ebc4621 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:47.987084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f5488816-5536-44fa-876b-897b6ebc4621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-eee982a7-d0f7-4187-b3cc-9d5a51f7c30e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:51:47.991166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:47.992040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:47.992936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 245/488] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jun 20 10:51:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:47.994015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ce887-58bd-4c6d-842c-8b69fcf4b4fd None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:47.994332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ce887-58bd-4c6d-842c-8b69fcf4b4fd None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:47.994607 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ce887-58bd-4c6d-842c-8b69fcf4b4fd None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:47.996152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 246/489] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:48.000986 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ce887-58bd-4c6d-842c-8b69fcf4b4fd None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:48.049052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:48.069911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 244/490] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:48.069911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db333c-ace9-4d5a-915e-ed36e30a005d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.069911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db333c-ace9-4d5a-915e-ed36e30a005d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.069911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db333c-ace9-4d5a-915e-ed36e30a005d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.086887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-01db333c-ace9-4d5a-915e-ed36e30a005d None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:48.088288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-01db333c-ace9-4d5a-915e-ed36e30a005d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:48.124111 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:48.146184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ce887-58bd-4c6d-842c-8b69fcf4b4fd None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:48.172330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 245/491] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:48.182952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db333c-ace9-4d5a-915e-ed36e30a005d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:48.184974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c48dc-666b-497c-be1c-a3e2031bae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:48.185643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c48dc-666b-497c-be1c-a3e2031bae44 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.185985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c48dc-666b-497c-be1c-a3e2031bae44 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.186223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c48dc-666b-497c-be1c-a3e2031bae44 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.187058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c48dc-666b-497c-be1c-a3e2031bae44 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:48.187372 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c48dc-666b-497c-be1c-a3e2031bae44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:48.188815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c48dc-666b-497c-be1c-a3e2031bae44 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:48.226225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 247/492] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jun 20 10:51:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:48.227479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55dbc1-2eed-4aee-bc3a-bca754783e72 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.227768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55dbc1-2eed-4aee-bc3a-bca754783e72 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.228041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55dbc1-2eed-4aee-bc3a-bca754783e72 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.233471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 248/493] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:48.233471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55dbc1-2eed-4aee-bc3a-bca754783e72 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:48.294531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db333c-ace9-4d5a-915e-ed36e30a005d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:48.313600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:48.333633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7128ae4-fc7c-45a0-8c9f-688772393eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:48.334480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128ae4-fc7c-45a0-8c9f-688772393eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.334791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55dbc1-2eed-4aee-bc3a-bca754783e72 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:48.334894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128ae4-fc7c-45a0-8c9f-688772393eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.335070 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7128ae4-fc7c-45a0-8c9f-688772393eb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.349000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7128ae4-fc7c-45a0-8c9f-688772393eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-e7128ae4-fc7c-45a0-8c9f-688772393eb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:48.357376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:48.358303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:48.359201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 246/494] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:48.359853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7128ae4-fc7c-45a0-8c9f-688772393eb5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:48.374000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a277bdd-61e5-4cb2-80bd-bcf1b5807f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:48.375775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a277bdd-61e5-4cb2-80bd-bcf1b5807f0c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.376249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a277bdd-61e5-4cb2-80bd-bcf1b5807f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.376709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a277bdd-61e5-4cb2-80bd-bcf1b5807f0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.391255 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a277bdd-61e5-4cb2-80bd-bcf1b5807f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6a277bdd-61e5-4cb2-80bd-bcf1b5807f0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:48.401397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:48.402497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:48.403407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:48.403997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 249/495] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:48.404889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6dc7-1cf4-4aef-9554-c381b73b70f3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.405175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6dc7-1cf4-4aef-9554-c381b73b70f3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.405450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c6dc7-1cf4-4aef-9554-c381b73b70f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.410732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:48.426362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c39c6dc7-1cf4-4aef-9554-c381b73b70f3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:48.427568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c39c6dc7-1cf4-4aef-9554-c381b73b70f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:48.470902 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7128ae4-fc7c-45a0-8c9f-688772393eb5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:48.504492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a082a50-1196-46c1-9354-1ef2e65d7c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:48.505070 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082a50-1196-46c1-9354-1ef2e65d7c22 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.505348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082a50-1196-46c1-9354-1ef2e65d7c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.505628 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082a50-1196-46c1-9354-1ef2e65d7c22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.510459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 250/496] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:48.511289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6dc7-1cf4-4aef-9554-c381b73b70f3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:48.558127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:48.583415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a082a50-1196-46c1-9354-1ef2e65d7c22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:48.585132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a082a50-1196-46c1-9354-1ef2e65d7c22 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:48.622701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c6dc7-1cf4-4aef-9554-c381b73b70f3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:48.653028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:48.658073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ee970-3159-4041-a74c-27763820dd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:48.658779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ee970-3159-4041-a74c-27763820dd1f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.659059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ee970-3159-4041-a74c-27763820dd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.659333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ee970-3159-4041-a74c-27763820dd1f None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.660083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ee970-3159-4041-a74c-27763820dd1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:48.660392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ee970-3159-4041-a74c-27763820dd1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:48.661772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ee970-3159-4041-a74c-27763820dd1f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:48.670524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 251/497] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:48.671210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ee970-3159-4041-a74c-27763820dd1f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:48.697533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 247/498] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:48.698245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a082a50-1196-46c1-9354-1ef2e65d7c22 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:48.789745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:48.790492 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ee970-3159-4041-a74c-27763820dd1f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:48.813760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a082a50-1196-46c1-9354-1ef2e65d7c22 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:48.825587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db509e1-ff5b-42cb-8064-e9b03185f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:48.826287 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db509e1-ff5b-42cb-8064-e9b03185f9ff None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.826565 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db509e1-ff5b-42cb-8064-e9b03185f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.826835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db509e1-ff5b-42cb-8064-e9b03185f9ff None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.827579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db509e1-ff5b-42cb-8064-e9b03185f9ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:48.827885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db509e1-ff5b-42cb-8064-e9b03185f9ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:48.829276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db509e1-ff5b-42cb-8064-e9b03185f9ff None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:48.837780 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 252/499] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:48.842673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db509e1-ff5b-42cb-8064-e9b03185f9ff None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:48.855220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8cfd5-b507-44bf-9df2-e7b6935c1e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:48.856162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8cfd5-b507-44bf-9df2-e7b6935c1e48 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.856438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8cfd5-b507-44bf-9df2-e7b6935c1e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.856711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8cfd5-b507-44bf-9df2-e7b6935c1e48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.868529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8cfd5-b507-44bf-9df2-e7b6935c1e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-99a8cfd5-b507-44bf-9df2-e7b6935c1e48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:48.876506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:48.877377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 248/500] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:48.878645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8cfd5-b507-44bf-9df2-e7b6935c1e48 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:48.901323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:48.945822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db509e1-ff5b-42cb-8064-e9b03185f9ff None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:48.979451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7890c58-92f0-485f-abd9-227c6d10b158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:48.980307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8cfd5-b507-44bf-9df2-e7b6935c1e48 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:48.980724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7890c58-92f0-485f-abd9-227c6d10b158 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:48.980724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7890c58-92f0-485f-abd9-227c6d10b158 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:48.980989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7890c58-92f0-485f-abd9-227c6d10b158 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:48.991293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7890c58-92f0-485f-abd9-227c6d10b158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:48.992743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7890c58-92f0-485f-abd9-227c6d10b158 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d7890c58-92f0-485f-abd9-227c6d10b158 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:49.009168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:49.010024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:49.010915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:49.010915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:49.010915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:49.010915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:49.010915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 253/501] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:49.011191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7890c58-92f0-485f-abd9-227c6d10b158 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.014590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4f0db-5f8a-4c60-9330-55222d206e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.015597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f0db-5f8a-4c60-9330-55222d206e05 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.015886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f0db-5f8a-4c60-9330-55222d206e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.016167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f0db-5f8a-4c60-9330-55222d206e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.023131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:49.027995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4f0db-5f8a-4c60-9330-55222d206e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.029886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4f0db-5f8a-4c60-9330-55222d206e05 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-e6e4f0db-5f8a-4c60-9330-55222d206e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:49.048341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:49.049366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:49.050379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:49.050379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:49.050379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:49.050379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:49.050379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 249/502] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:49.051057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f42be-6fa6-49f0-86f3-bfdaebefc248 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.051153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f42be-6fa6-49f0-86f3-bfdaebefc248 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.051390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f42be-6fa6-49f0-86f3-bfdaebefc248 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.054177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 250/503] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:49.054177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f42be-6fa6-49f0-86f3-bfdaebefc248 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.121830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7890c58-92f0-485f-abd9-227c6d10b158 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.130481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:49.156069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f42be-6fa6-49f0-86f3-bfdaebefc248 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.156477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fd7c2-70fb-4f55-a8ee-d77d468a78d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.157293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fd7c2-70fb-4f55-a8ee-d77d468a78d4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.157572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fd7c2-70fb-4f55-a8ee-d77d468a78d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.157845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fd7c2-70fb-4f55-a8ee-d77d468a78d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.168577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fd7c2-70fb-4f55-a8ee-d77d468a78d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.170061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fd7c2-70fb-4f55-a8ee-d77d468a78d4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-988fd7c2-70fb-4f55-a8ee-d77d468a78d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:49.187520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:49.188357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:49.189191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:49.189191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:49.189191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:49.189191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:49.189191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 254/504] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:49.189513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fd7c2-70fb-4f55-a8ee-d77d468a78d4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.194039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8648b9-cc79-43ab-91e0-086cfdde961a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.194742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8648b9-cc79-43ab-91e0-086cfdde961a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.195036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8648b9-cc79-43ab-91e0-086cfdde961a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.195313 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8648b9-cc79-43ab-91e0-086cfdde961a None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.196075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8648b9-cc79-43ab-91e0-086cfdde961a None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:49.196385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8648b9-cc79-43ab-91e0-086cfdde961a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.197776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8648b9-cc79-43ab-91e0-086cfdde961a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.206507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 251/505] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:49.211327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8648b9-cc79-43ab-91e0-086cfdde961a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.260545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:49.293312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fd7c2-70fb-4f55-a8ee-d77d468a78d4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.308225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8648b9-cc79-43ab-91e0-086cfdde961a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.326291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bcfc9-cfed-41e8-ad3b-08c71d51944f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.326953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bcfc9-cfed-41e8-ad3b-08c71d51944f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.327235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bcfc9-cfed-41e8-ad3b-08c71d51944f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.327506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bcfc9-cfed-41e8-ad3b-08c71d51944f None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.328223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bcfc9-cfed-41e8-ad3b-08c71d51944f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:49.328533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bcfc9-cfed-41e8-ad3b-08c71d51944f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.329928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bcfc9-cfed-41e8-ad3b-08c71d51944f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.341190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30995fc3-1f0d-4370-9827-b777bcd9b0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.341848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30995fc3-1f0d-4370-9827-b777bcd9b0cb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.342156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30995fc3-1f0d-4370-9827-b777bcd9b0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.342247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 255/506] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:49.342434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30995fc3-1f0d-4370-9827-b777bcd9b0cb None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.343147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30995fc3-1f0d-4370-9827-b777bcd9b0cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:49.343388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3ef3-bba8-4a7f-b39f-337d57ecce62 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.343489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30995fc3-1f0d-4370-9827-b777bcd9b0cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.343671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3ef3-bba8-4a7f-b39f-337d57ecce62 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.343926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e3ef3-bba8-4a7f-b39f-337d57ecce62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.344805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30995fc3-1f0d-4370-9827-b777bcd9b0cb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.356779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 252/507] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:49.357572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30995fc3-1f0d-4370-9827-b777bcd9b0cb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.361897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-153e3ef3-bba8-4a7f-b39f-337d57ecce62 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:49.363122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-153e3ef3-bba8-4a7f-b39f-337d57ecce62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:49.377732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:49.430025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 256/508] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:49.431102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3ef3-bba8-4a7f-b39f-337d57ecce62 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.464547 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30995fc3-1f0d-4370-9827-b777bcd9b0cb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.478970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:49.500654 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6239-aba9-4676-9867-319d66c46a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.501412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6239-aba9-4676-9867-319d66c46a6e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.501759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6239-aba9-4676-9867-319d66c46a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.502111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6239-aba9-4676-9867-319d66c46a6e None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.502927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6239-aba9-4676-9867-319d66c46a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:49.503320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6239-aba9-4676-9867-319d66c46a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.504787 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f6239-aba9-4676-9867-319d66c46a6e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.516294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 253/509] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:49.521087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6239-aba9-4676-9867-319d66c46a6e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.527674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e3ef3-bba8-4a7f-b39f-337d57ecce62 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.560575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748ed45-0516-4bc8-b668-b0c2fbda6d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.561303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748ed45-0516-4bc8-b668-b0c2fbda6d7c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.561582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748ed45-0516-4bc8-b668-b0c2fbda6d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.561851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748ed45-0516-4bc8-b668-b0c2fbda6d7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.571651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748ed45-0516-4bc8-b668-b0c2fbda6d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.573131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748ed45-0516-4bc8-b668-b0c2fbda6d7c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-f748ed45-0516-4bc8-b668-b0c2fbda6d7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:49.590608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:49.591496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:49.592327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:49.592327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:49.592327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:49.592327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:49.592327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 257/510] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:49.592665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748ed45-0516-4bc8-b668-b0c2fbda6d7c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.595325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:49.616054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6239-aba9-4676-9867-319d66c46a6e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.648336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e7bd4-b6d6-4f87-a929-193314c0cbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.649134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e7bd4-b6d6-4f87-a929-193314c0cbdc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.649541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e7bd4-b6d6-4f87-a929-193314c0cbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.649687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e7bd4-b6d6-4f87-a929-193314c0cbdc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.657629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e7bd4-b6d6-4f87-a929-193314c0cbdc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.659133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e7bd4-b6d6-4f87-a929-193314c0cbdc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-267e7bd4-b6d6-4f87-a929-193314c0cbdc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:51:49.674885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:49.675863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:49.676728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 254/511] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:49.677448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e7bd4-b6d6-4f87-a929-193314c0cbdc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.695670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748ed45-0516-4bc8-b668-b0c2fbda6d7c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.709775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:49.732060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141bb9b-27de-4649-a723-692cf945297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.732850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141bb9b-27de-4649-a723-692cf945297d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.733127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141bb9b-27de-4649-a723-692cf945297d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.733497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141bb9b-27de-4649-a723-692cf945297d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.740490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141bb9b-27de-4649-a723-692cf945297d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.741931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141bb9b-27de-4649-a723-692cf945297d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a141bb9b-27de-4649-a723-692cf945297d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:51:49.756460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:49.757312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:49.758201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:51:49.758201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:51:49.758201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:51:49.758201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:51:49.758201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:51:49.758201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:51:49.758201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:49.758201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:49.758201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 258/512] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:49.758790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141bb9b-27de-4649-a723-692cf945297d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.782965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e7bd4-b6d6-4f87-a929-193314c0cbdc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.816417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543e254-e3e3-4e2b-a2b0-d98053f9ecf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.817147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543e254-e3e3-4e2b-a2b0-d98053f9ecf5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.817426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543e254-e3e3-4e2b-a2b0-d98053f9ecf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.817694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543e254-e3e3-4e2b-a2b0-d98053f9ecf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.821056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:49.828457 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543e254-e3e3-4e2b-a2b0-d98053f9ecf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.829928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543e254-e3e3-4e2b-a2b0-d98053f9ecf5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d543e254-e3e3-4e2b-a2b0-d98053f9ecf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:49.847641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:49.848512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:49.849363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:49.849363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:49.849363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:49.849363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:49.849363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 255/513] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:49.849626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543e254-e3e3-4e2b-a2b0-d98053f9ecf5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.863390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141bb9b-27de-4649-a723-692cf945297d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.897306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf403b-a82a-45d6-8e4d-04bf0cffd872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.898119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf403b-a82a-45d6-8e4d-04bf0cffd872 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.898396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf403b-a82a-45d6-8e4d-04bf0cffd872 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.898664 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf403b-a82a-45d6-8e4d-04bf0cffd872 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.906620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf403b-a82a-45d6-8e4d-04bf0cffd872 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.908103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf403b-a82a-45d6-8e4d-04bf0cffd872 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-64bf403b-a82a-45d6-8e4d-04bf0cffd872 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:51:49.923182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:49.924286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 259/514] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:49.925392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf403b-a82a-45d6-8e4d-04bf0cffd872 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:49.943384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:49.959035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543e254-e3e3-4e2b-a2b0-d98053f9ecf5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:49.991426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fa289-795b-4d14-b1c1-eef13f0cb2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:49.992106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fa289-795b-4d14-b1c1-eef13f0cb2ef None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:49.992368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fa289-795b-4d14-b1c1-eef13f0cb2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:49.992638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fa289-795b-4d14-b1c1-eef13f0cb2ef None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:49.993357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fa289-795b-4d14-b1c1-eef13f0cb2ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:49.993661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fa289-795b-4d14-b1c1-eef13f0cb2ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:49.995083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fa289-795b-4d14-b1c1-eef13f0cb2ef None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:50.006760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 256/515] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:50.007431 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fa289-795b-4d14-b1c1-eef13f0cb2ef None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:50.024960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf403b-a82a-45d6-8e4d-04bf0cffd872 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:50.048987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:50.061256 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e239a-e80f-419b-9bea-1fa65553e28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:50.061913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e239a-e80f-419b-9bea-1fa65553e28b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:50.062235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e239a-e80f-419b-9bea-1fa65553e28b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:50.062519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e239a-e80f-419b-9bea-1fa65553e28b None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:50.063369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e239a-e80f-419b-9bea-1fa65553e28b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:50.063685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e239a-e80f-419b-9bea-1fa65553e28b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:50.065159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e239a-e80f-419b-9bea-1fa65553e28b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:50.083938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 260/516] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jun 20 10:51:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:50.084971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e239a-e80f-419b-9bea-1fa65553e28b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:50.114939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fa289-795b-4d14-b1c1-eef13f0cb2ef None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:50.151274 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add48aa9-c2bf-4b08-898b-a8a36c22712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:50.152019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add48aa9-c2bf-4b08-898b-a8a36c22712f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:50.152840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add48aa9-c2bf-4b08-898b-a8a36c22712f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:50.153097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add48aa9-c2bf-4b08-898b-a8a36c22712f None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:50.153920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add48aa9-c2bf-4b08-898b-a8a36c22712f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:50.154266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add48aa9-c2bf-4b08-898b-a8a36c22712f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:50.155728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add48aa9-c2bf-4b08-898b-a8a36c22712f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:50.174349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 257/517] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:51:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:50.174964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add48aa9-c2bf-4b08-898b-a8a36c22712f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:50.178411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:50.203017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e239a-e80f-419b-9bea-1fa65553e28b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:50.237654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ced2a-c7ed-4328-8aa5-1675331a3071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:50.238377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ced2a-c7ed-4328-8aa5-1675331a3071 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:50.238663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ced2a-c7ed-4328-8aa5-1675331a3071 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:50.238938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ced2a-c7ed-4328-8aa5-1675331a3071 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:50.239666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ced2a-c7ed-4328-8aa5-1675331a3071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:50.239975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ced2a-c7ed-4328-8aa5-1675331a3071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:50.241378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ced2a-c7ed-4328-8aa5-1675331a3071 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:50.256744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 261/518] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:50.256744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ced2a-c7ed-4328-8aa5-1675331a3071 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:50.283465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add48aa9-c2bf-4b08-898b-a8a36c22712f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:50.296604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:50.316817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d4d52-9fe6-4fe0-ad3b-b4f0a7390873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:50.317508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d4d52-9fe6-4fe0-ad3b-b4f0a7390873 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:50.317787 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d4d52-9fe6-4fe0-ad3b-b4f0a7390873 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:50.318039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d4d52-9fe6-4fe0-ad3b-b4f0a7390873 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:50.318904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d4d52-9fe6-4fe0-ad3b-b4f0a7390873 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:50.319211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d4d52-9fe6-4fe0-ad3b-b4f0a7390873 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:50.320672 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d4d52-9fe6-4fe0-ad3b-b4f0a7390873 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:50.356212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ced2a-c7ed-4328-8aa5-1675331a3071 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:50.356382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 258/519] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Jun 20 10:51:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:50.357019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d4d52-9fe6-4fe0-ad3b-b4f0a7390873 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:50.391483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa28a0-0862-4d71-9af0-5348f1caa30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:50.392310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa28a0-0862-4d71-9af0-5348f1caa30b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:50.392589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa28a0-0862-4d71-9af0-5348f1caa30b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:50.392861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa28a0-0862-4d71-9af0-5348f1caa30b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:50.400510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa28a0-0862-4d71-9af0-5348f1caa30b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:50.401971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa28a0-0862-4d71-9af0-5348f1caa30b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e8aa28a0-0862-4d71-9af0-5348f1caa30b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:51:50.416893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:50.417991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 262/520] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:50.424397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa28a0-0862-4d71-9af0-5348f1caa30b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:50.425015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:50.463216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d4d52-9fe6-4fe0-ad3b-b4f0a7390873 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:50.498050 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd39ac3-37ed-4e4e-9f53-26b129b2c7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:50.498723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd39ac3-37ed-4e4e-9f53-26b129b2c7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:50.499001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd39ac3-37ed-4e4e-9f53-26b129b2c7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:50.499272 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd39ac3-37ed-4e4e-9f53-26b129b2c7c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:50.519440 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:50.522794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa28a0-0862-4d71-9af0-5348f1caa30b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:50.554835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e07c4-2ccd-48c4-8a36-993bcb5e9d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:50.555478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e07c4-2ccd-48c4-8a36-993bcb5e9d71 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:50.555756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e07c4-2ccd-48c4-8a36-993bcb5e9d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:50.556028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e07c4-2ccd-48c4-8a36-993bcb5e9d71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:50.556853 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e07c4-2ccd-48c4-8a36-993bcb5e9d71 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=37b1f13aa474493d83b3a16f76949fbc) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:50.557179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e07c4-2ccd-48c4-8a36-993bcb5e9d71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:50.558835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e07c4-2ccd-48c4-8a36-993bcb5e9d71 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:50.564068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 263/521] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:50 2022] GET /identity/v3/role_assignments?user.id=37b1f13aa474493d83b3a16f76949fbc&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:50.564804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e07c4-2ccd-48c4-8a36-993bcb5e9d71 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:50.581681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd39ac3-37ed-4e4e-9f53-26b129b2c7c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:50.583651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd39ac3-37ed-4e4e-9f53-26b129b2c7c9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:50.658162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:50.680468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e07c4-2ccd-48c4-8a36-993bcb5e9d71 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:50.704203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 259/522] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:50.705329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd39ac3-37ed-4e4e-9f53-26b129b2c7c9 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:50.726301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ba235-cd69-42dc-98ef-4cc5dc507ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:50.726984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ba235-cd69-42dc-98ef-4cc5dc507ef4 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:50.727892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ba235-cd69-42dc-98ef-4cc5dc507ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:50.727892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ba235-cd69-42dc-98ef-4cc5dc507ef4 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:50.728270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ba235-cd69-42dc-98ef-4cc5dc507ef4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:50.729726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ba235-cd69-42dc-98ef-4cc5dc507ef4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:50.755290 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 264/523] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:51:50 2022] POST /identity/v3/users => generated 247 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:51:50.756302 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ba235-cd69-42dc-98ef-4cc5dc507ef4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:50.789474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:50.826175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd39ac3-37ed-4e4e-9f53-26b129b2c7c9 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:50.875513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d47d4-df17-4e2f-96d6-424e05b4ae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:50.876243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d47d4-df17-4e2f-96d6-424e05b4ae63 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:50.876634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d47d4-df17-4e2f-96d6-424e05b4ae63 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:50.876917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d47d4-df17-4e2f-96d6-424e05b4ae63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:50.904189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ba235-cd69-42dc-98ef-4cc5dc507ef4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:50.924145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:50.943283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ef6e1-b428-4c0a-ade6-a40413b9090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:50.943969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef6e1-b428-4c0a-ade6-a40413b9090e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:50.944249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef6e1-b428-4c0a-ade6-a40413b9090e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:50.944520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef6e1-b428-4c0a-ade6-a40413b9090e None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:50.945367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ef6e1-b428-4c0a-ade6-a40413b9090e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:50.945717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ef6e1-b428-4c0a-ade6-a40413b9090e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:50.947189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ef6e1-b428-4c0a-ade6-a40413b9090e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:50.965795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 265/524] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jun 20 10:51:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:50.966774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ef6e1-b428-4c0a-ade6-a40413b9090e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:50.969596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d47d4-df17-4e2f-96d6-424e05b4ae63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:50.971462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d47d4-df17-4e2f-96d6-424e05b4ae63 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:51.067587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:51.084748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 260/525] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:51.086286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d47d4-df17-4e2f-96d6-424e05b4ae63 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:51.098904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ef6e1-b428-4c0a-ade6-a40413b9090e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:51.143578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c897a5e-bb3a-4e71-a6ea-e4ef827ccb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:51.144562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c897a5e-bb3a-4e71-a6ea-e4ef827ccb91 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:51.144849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c897a5e-bb3a-4e71-a6ea-e4ef827ccb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:51.145124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c897a5e-bb3a-4e71-a6ea-e4ef827ccb91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:51.156184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:51.160071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c897a5e-bb3a-4e71-a6ea-e4ef827ccb91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:51.161890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c897a5e-bb3a-4e71-a6ea-e4ef827ccb91 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1c897a5e-bb3a-4e71-a6ea-e4ef827ccb91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:51.185948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:51.186994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:51.187847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:51.187847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:51.187847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:51.187847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:51.187847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 266/526] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:51.193185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c897a5e-bb3a-4e71-a6ea-e4ef827ccb91 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:51.205707 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d47d4-df17-4e2f-96d6-424e05b4ae63 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:51.253118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e63d0-1058-4828-adf8-ae6a8b638866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:51.253785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e63d0-1058-4828-adf8-ae6a8b638866 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:51.254070 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e63d0-1058-4828-adf8-ae6a8b638866 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:51.254373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e63d0-1058-4828-adf8-ae6a8b638866 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:51.303644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c897a5e-bb3a-4e71-a6ea-e4ef827ccb91 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:51.337084 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:51.345153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e63d0-1058-4828-adf8-ae6a8b638866 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:51.346234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbf4b-cae5-4ee8-ab4b-0e6847bc9a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:51.346758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e63d0-1058-4828-adf8-ae6a8b638866 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:51.347515 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbf4b-cae5-4ee8-ab4b-0e6847bc9a8c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:51.347993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbf4b-cae5-4ee8-ab4b-0e6847bc9a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:51.348554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cbf4b-cae5-4ee8-ab4b-0e6847bc9a8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:51.363855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cbf4b-cae5-4ee8-ab4b-0e6847bc9a8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:51.365743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cbf4b-cae5-4ee8-ab4b-0e6847bc9a8c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-f18cbf4b-cae5-4ee8-ab4b-0e6847bc9a8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:51.385228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:51.386126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:51.387063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:51.387063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:51.387063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:51.387063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:51.389398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 267/527] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:51.390421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbf4b-cae5-4ee8-ab4b-0e6847bc9a8c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:51.422258 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:51.474434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 261/528] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:51.475617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e63d0-1058-4828-adf8-ae6a8b638866 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:51.508723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cbf4b-cae5-4ee8-ab4b-0e6847bc9a8c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:51.545146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fa12c-4e71-4f51-ba88-834f443cba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:51.546214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fa12c-4e71-4f51-ba88-834f443cba49 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:51.546721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fa12c-4e71-4f51-ba88-834f443cba49 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:51.547189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fa12c-4e71-4f51-ba88-834f443cba49 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:51.548212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fa12c-4e71-4f51-ba88-834f443cba49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:51.548716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fa12c-4e71-4f51-ba88-834f443cba49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:51.550335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fa12c-4e71-4f51-ba88-834f443cba49 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:51.567103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 268/529] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:51.568301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fa12c-4e71-4f51-ba88-834f443cba49 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:51.606181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e63d0-1058-4828-adf8-ae6a8b638866 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:51.620388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:51.652435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3eded9-66de-420b-bb54-a17b837732f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:51.653110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3eded9-66de-420b-bb54-a17b837732f2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:51.653398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3eded9-66de-420b-bb54-a17b837732f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:51.653675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3eded9-66de-420b-bb54-a17b837732f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:51.654675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3eded9-66de-420b-bb54-a17b837732f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:51.654994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3eded9-66de-420b-bb54-a17b837732f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:51.657348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3eded9-66de-420b-bb54-a17b837732f2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:51.673745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 262/530] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:51.675975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3eded9-66de-420b-bb54-a17b837732f2 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:51.699244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fa12c-4e71-4f51-ba88-834f443cba49 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:51.712478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:51.736974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dac52-f776-4f98-8f2f-53ef24ee80da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:51.738158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dac52-f776-4f98-8f2f-53ef24ee80da None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:51.738673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dac52-f776-4f98-8f2f-53ef24ee80da None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:51.739165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dac52-f776-4f98-8f2f-53ef24ee80da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:51.750991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dac52-f776-4f98-8f2f-53ef24ee80da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:51.752788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dac52-f776-4f98-8f2f-53ef24ee80da None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-9f8dac52-f776-4f98-8f2f-53ef24ee80da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:51.772884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:51.773741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:51.774629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:51.774629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:51.774629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:51.774629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:51.776532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 269/531] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:51.777578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dac52-f776-4f98-8f2f-53ef24ee80da None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:51.815708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3eded9-66de-420b-bb54-a17b837732f2 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:51.860037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46281f11-451b-42ca-be1e-59e0bee8610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:51.860853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46281f11-451b-42ca-be1e-59e0bee8610a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:51.861664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46281f11-451b-42ca-be1e-59e0bee8610a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:51.861965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46281f11-451b-42ca-be1e-59e0bee8610a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:51.871251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46281f11-451b-42ca-be1e-59e0bee8610a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:51.873529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46281f11-451b-42ca-be1e-59e0bee8610a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:51.881642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:51.889921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dac52-f776-4f98-8f2f-53ef24ee80da None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-46281f11-451b-42ca-be1e-59e0bee8610a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:51:51.895458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:51.896289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:51.897119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:51:51.897119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:51:51.897119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:51:51.897119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:51:51.897119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:51:51.897119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:51:51.897119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:51.897119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:51.897119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 263/532] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:51.902610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46281f11-451b-42ca-be1e-59e0bee8610a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:51.938402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca393d32-5600-4e63-b0a0-3731dc46a306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:51.939217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca393d32-5600-4e63-b0a0-3731dc46a306 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:51.939502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca393d32-5600-4e63-b0a0-3731dc46a306 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:51.939771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca393d32-5600-4e63-b0a0-3731dc46a306 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:51.947435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca393d32-5600-4e63-b0a0-3731dc46a306 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:51.949108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca393d32-5600-4e63-b0a0-3731dc46a306 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-ca393d32-5600-4e63-b0a0-3731dc46a306 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:51:51.965845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:51.966792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:51.967625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:51:51.967625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:51:51.967625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:51:51.967625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:51:51.967625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:51:51.967625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:51:51.967625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:51.967625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:51.967625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 270/533] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:51.968540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca393d32-5600-4e63-b0a0-3731dc46a306 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:52.014746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:52.034423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46281f11-451b-42ca-be1e-59e0bee8610a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:52.069546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33f7a7-9b70-46e9-9b33-93430f86d138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:52.070448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33f7a7-9b70-46e9-9b33-93430f86d138 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.070829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33f7a7-9b70-46e9-9b33-93430f86d138 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.071200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33f7a7-9b70-46e9-9b33-93430f86d138 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.072029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33f7a7-9b70-46e9-9b33-93430f86d138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:52.072436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33f7a7-9b70-46e9-9b33-93430f86d138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:52.074036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33f7a7-9b70-46e9-9b33-93430f86d138 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:52.090699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 264/534] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:52.091718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33f7a7-9b70-46e9-9b33-93430f86d138 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:52.097224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca393d32-5600-4e63-b0a0-3731dc46a306 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:52.145616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080dd913-56ce-4f6c-b15c-30a50b259dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:52.146424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dd913-56ce-4f6c-b15c-30a50b259dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.146708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dd913-56ce-4f6c-b15c-30a50b259dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.147015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dd913-56ce-4f6c-b15c-30a50b259dc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.148289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080dd913-56ce-4f6c-b15c-30a50b259dc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:52.148868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080dd913-56ce-4f6c-b15c-30a50b259dc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:52.150997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080dd913-56ce-4f6c-b15c-30a50b259dc4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:52.170379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 271/535] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:51:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:52.171416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080dd913-56ce-4f6c-b15c-30a50b259dc4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:52.188001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:52.209049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33f7a7-9b70-46e9-9b33-93430f86d138 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:52.252735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966ad00-3265-40dc-8014-7d91900d8be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:52.253917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966ad00-3265-40dc-8014-7d91900d8be3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.254237 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966ad00-3265-40dc-8014-7d91900d8be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.254517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966ad00-3265-40dc-8014-7d91900d8be3 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.255319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966ad00-3265-40dc-8014-7d91900d8be3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:52.256077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966ad00-3265-40dc-8014-7d91900d8be3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:52.257966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966ad00-3265-40dc-8014-7d91900d8be3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:52.278252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 265/536] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:51:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:52.279637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966ad00-3265-40dc-8014-7d91900d8be3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:52.282617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:52.284303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080dd913-56ce-4f6c-b15c-30a50b259dc4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:52.329589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350b394-c0c5-454f-a91c-2910dbecb024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:52.330825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350b394-c0c5-454f-a91c-2910dbecb024 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.331111 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350b394-c0c5-454f-a91c-2910dbecb024 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.331388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350b394-c0c5-454f-a91c-2910dbecb024 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.341303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350b394-c0c5-454f-a91c-2910dbecb024 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:52.343050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350b394-c0c5-454f-a91c-2910dbecb024 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-f350b394-c0c5-454f-a91c-2910dbecb024 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:51:52.361027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:52.361998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:52.367770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:51:52.367770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:51:52.367770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:51:52.367770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:51:52.367770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:51:52.367770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:51:52.367770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:52.367770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:52.367770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 272/537] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:52.367770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350b394-c0c5-454f-a91c-2910dbecb024 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:52.395153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966ad00-3265-40dc-8014-7d91900d8be3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:52.433559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281bb53-582a-4a0a-a51d-8bc6aa87b3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:52.434253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281bb53-582a-4a0a-a51d-8bc6aa87b3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.434540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281bb53-582a-4a0a-a51d-8bc6aa87b3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.434816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281bb53-582a-4a0a-a51d-8bc6aa87b3e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.435779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281bb53-582a-4a0a-a51d-8bc6aa87b3e9 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=841e89a6caef4a089edc0f3b1df2fe9e) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:52.436015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281bb53-582a-4a0a-a51d-8bc6aa87b3e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:52.437532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281bb53-582a-4a0a-a51d-8bc6aa87b3e9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:52.443952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 266/538] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:52 2022] GET /identity/v3/role_assignments?user.id=841e89a6caef4a089edc0f3b1df2fe9e&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:52.444794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281bb53-582a-4a0a-a51d-8bc6aa87b3e9 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:52.461410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:52.495745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350b394-c0c5-454f-a91c-2910dbecb024 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:52.521496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:52.541040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca8312-feb9-4f3e-8ad2-6f7774f3568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:52.542151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca8312-feb9-4f3e-8ad2-6f7774f3568c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.542541 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca8312-feb9-4f3e-8ad2-6f7774f3568c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.543009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca8312-feb9-4f3e-8ad2-6f7774f3568c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.544293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca8312-feb9-4f3e-8ad2-6f7774f3568c None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=d73fe529c7d342f895aabcb9a39aba78) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:52.544708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca8312-feb9-4f3e-8ad2-6f7774f3568c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:52.546820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca8312-feb9-4f3e-8ad2-6f7774f3568c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:52.555219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 273/539] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:52 2022] GET /identity/v3/role_assignments?user.id=d73fe529c7d342f895aabcb9a39aba78&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:52.580902 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281bb53-582a-4a0a-a51d-8bc6aa87b3e9 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:52.601424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072c6d0-fb02-4589-bc5b-3a32806bec5b None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.601743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072c6d0-fb02-4589-bc5b-3a32806bec5b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.601975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072c6d0-fb02-4589-bc5b-3a32806bec5b None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.603579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 274/540] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:51:52.609468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ba6f-ed8a-4e53-9869-a24761597d48 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.610069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ba6f-ed8a-4e53-9869-a24761597d48 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.610567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ba6f-ed8a-4e53-9869-a24761597d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.631556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a0c3ba6f-ed8a-4e53-9869-a24761597d48 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:52.633190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a0c3ba6f-ed8a-4e53-9869-a24761597d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:52.652638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afb035-0fc5-4429-840d-161c98a62f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:52.653344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afb035-0fc5-4429-840d-161c98a62f45 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.653633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afb035-0fc5-4429-840d-161c98a62f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.653917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afb035-0fc5-4429-840d-161c98a62f45 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.654746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afb035-0fc5-4429-840d-161c98a62f45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:52.655061 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afb035-0fc5-4429-840d-161c98a62f45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:52.656673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afb035-0fc5-4429-840d-161c98a62f45 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:52.680176 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 267/541] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:51:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:52.690145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afb035-0fc5-4429-840d-161c98a62f45 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:52.752183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 275/542] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:52.781087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3ba6f-ed8a-4e53-9869-a24761597d48 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:52.829608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afb035-0fc5-4429-840d-161c98a62f45 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:52.858805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:52.860358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:52.880737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd8ef0-99ba-4c81-87b0-f191f7d74398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:52.881579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd8ef0-99ba-4c81-87b0-f191f7d74398 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.882606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd8ef0-99ba-4c81-87b0-f191f7d74398 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.882961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd8ef0-99ba-4c81-87b0-f191f7d74398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.883936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd8ef0-99ba-4c81-87b0-f191f7d74398 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=e4bf4d9571d541bcbba87094198dc2ff) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:52.884230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd8ef0-99ba-4c81-87b0-f191f7d74398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:52.885862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd8ef0-99ba-4c81-87b0-f191f7d74398 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:52.895898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3ba6f-ed8a-4e53-9869-a24761597d48 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:52.901755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 268/543] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:52 2022] GET /identity/v3/role_assignments?user.id=e4bf4d9571d541bcbba87094198dc2ff&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:52.963711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ec4ad-2b0d-4d7d-b1d7-13685167d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:52.965100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ec4ad-2b0d-4d7d-b1d7-13685167d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.965569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ec4ad-2b0d-4d7d-b1d7-13685167d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.966017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ec4ad-2b0d-4d7d-b1d7-13685167d7b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.982127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ec4ad-2b0d-4d7d-b1d7-13685167d7b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-549ec4ad-2b0d-4d7d-b1d7-13685167d7b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:52.990602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:52.991518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:52.992323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:52.993100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d3051-711a-4e43-8735-edbb785aebc3 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:52.993386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d3051-711a-4e43-8735-edbb785aebc3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:52.993666 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d3051-711a-4e43-8735-edbb785aebc3 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:52.995315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 276/545] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:52.995440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 269/545] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:53.006146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72a8bf-00f0-4e7f-a0a9-49bfb9c5ab82 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:53.006146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72a8bf-00f0-4e7f-a0a9-49bfb9c5ab82 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:53.006146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72a8bf-00f0-4e7f-a0a9-49bfb9c5ab82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:53.007189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ec4ad-2b0d-4d7d-b1d7-13685167d7b3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:53.039793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4c72a8bf-00f0-4e7f-a0a9-49bfb9c5ab82 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:53.041035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4c72a8bf-00f0-4e7f-a0a9-49bfb9c5ab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:53.153713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ec4ad-2b0d-4d7d-b1d7-13685167d7b3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:53.184083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:53.184865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 270/546] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:51:53.192961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7ac3d-7d32-4411-9722-c05ab0a76aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:53.193648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7ac3d-7d32-4411-9722-c05ab0a76aec None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:53.193976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7ac3d-7d32-4411-9722-c05ab0a76aec None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:53.194280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7ac3d-7d32-4411-9722-c05ab0a76aec None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:53.195034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7ac3d-7d32-4411-9722-c05ab0a76aec None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:53.195343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7ac3d-7d32-4411-9722-c05ab0a76aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:53.196794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7ac3d-7d32-4411-9722-c05ab0a76aec None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:53.205004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72a8bf-00f0-4e7f-a0a9-49bfb9c5ab82 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:53.209006 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 277/547] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:53.214173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7ac3d-7d32-4411-9722-c05ab0a76aec None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:53.343457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7ac3d-7d32-4411-9722-c05ab0a76aec None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:53.380821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b750-b563-4b86-bd64-573cb205cf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:53.382075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b750-b563-4b86-bd64-573cb205cf55 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:53.382617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b750-b563-4b86-bd64-573cb205cf55 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:53.383959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b750-b563-4b86-bd64-573cb205cf55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:53.396371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5b750-b563-4b86-bd64-573cb205cf55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:53.398257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5b750-b563-4b86-bd64-573cb205cf55 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-24b5b750-b563-4b86-bd64-573cb205cf55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:53.418409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:53.419284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:53.420130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:53.420130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:53.420130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:53.420130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:53.422685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 278/548] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:53.427753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b750-b563-4b86-bd64-573cb205cf55 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:53.476340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72a8bf-00f0-4e7f-a0a9-49bfb9c5ab82 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:53.508976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:53.510569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:53.539563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a40f5-bdcb-4ca9-a37e-16f982a9448d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:53.540375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a40f5-bdcb-4ca9-a37e-16f982a9448d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:53.540659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a40f5-bdcb-4ca9-a37e-16f982a9448d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:53.540933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a40f5-bdcb-4ca9-a37e-16f982a9448d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:53.559852 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a40f5-bdcb-4ca9-a37e-16f982a9448d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d67a40f5-bdcb-4ca9-a37e-16f982a9448d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:53.567577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:53.568531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:53.569867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:53.577766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 271/549] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:53.580076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a40f5-bdcb-4ca9-a37e-16f982a9448d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:53.598022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b750-b563-4b86-bd64-573cb205cf55 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:53.657044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6fa54-86df-4adb-8480-eb9eb63e473e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:53.657714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6fa54-86df-4adb-8480-eb9eb63e473e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:53.657996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6fa54-86df-4adb-8480-eb9eb63e473e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:53.658292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6fa54-86df-4adb-8480-eb9eb63e473e None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:53.659023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6fa54-86df-4adb-8480-eb9eb63e473e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:53.659328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6fa54-86df-4adb-8480-eb9eb63e473e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:53.660746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6fa54-86df-4adb-8480-eb9eb63e473e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:53.683632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 279/550] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:53.687226 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6fa54-86df-4adb-8480-eb9eb63e473e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:53.763877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a40f5-bdcb-4ca9-a37e-16f982a9448d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:53.827992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949ba75-1d74-4cc3-81d0-221074b9db12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:53.828675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949ba75-1d74-4cc3-81d0-221074b9db12 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:53.828956 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949ba75-1d74-4cc3-81d0-221074b9db12 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:53.829231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949ba75-1d74-4cc3-81d0-221074b9db12 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:53.829989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949ba75-1d74-4cc3-81d0-221074b9db12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:53.830335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949ba75-1d74-4cc3-81d0-221074b9db12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:53.833990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949ba75-1d74-4cc3-81d0-221074b9db12 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:53.858929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 272/551] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:53.860811 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949ba75-1d74-4cc3-81d0-221074b9db12 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:53.880068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6fa54-86df-4adb-8480-eb9eb63e473e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:53.881744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:53.938434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:53.944400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e66a5-2459-4be1-89c1-296e9b7788c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:53.946869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e66a5-2459-4be1-89c1-296e9b7788c2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:53.947588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e66a5-2459-4be1-89c1-296e9b7788c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:53.949119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e66a5-2459-4be1-89c1-296e9b7788c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:53.967391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e66a5-2459-4be1-89c1-296e9b7788c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:53.968869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e66a5-2459-4be1-89c1-296e9b7788c2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d94e66a5-2459-4be1-89c1-296e9b7788c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:51:53.998517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:53.999571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:54.000570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:54.000570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:54.000570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:51:54.000570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:51:54.000570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:51:54.000570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:51:54.000570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:51:54.000570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:51:54.000570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:51:54.000570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:54.007461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 280/552] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jun 20 10:51:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:54.013996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e66a5-2459-4be1-89c1-296e9b7788c2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:54.039280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949ba75-1d74-4cc3-81d0-221074b9db12 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:54.099696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3cbd1-56c8-4f82-97e3-aa01c46e47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:54.100862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3cbd1-56c8-4f82-97e3-aa01c46e47fe None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:54.101388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3cbd1-56c8-4f82-97e3-aa01c46e47fe None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:54.101856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3cbd1-56c8-4f82-97e3-aa01c46e47fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:54.121903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3cbd1-56c8-4f82-97e3-aa01c46e47fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:54.124461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3cbd1-56c8-4f82-97e3-aa01c46e47fe None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5ff3cbd1-56c8-4f82-97e3-aa01c46e47fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:54.158249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:54.159182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:54.160132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:54.160132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:54.160132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:54.160132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:54.168765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 273/553] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:54.172415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3cbd1-56c8-4f82-97e3-aa01c46e47fe None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:54.206876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e66a5-2459-4be1-89c1-296e9b7788c2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:54.246605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5069ce0-1b64-467f-9213-12fad2b4117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:54.247293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5069ce0-1b64-467f-9213-12fad2b4117d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:54.247573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5069ce0-1b64-467f-9213-12fad2b4117d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:54.247874 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5069ce0-1b64-467f-9213-12fad2b4117d None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:54.249184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5069ce0-1b64-467f-9213-12fad2b4117d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:54.249511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5069ce0-1b64-467f-9213-12fad2b4117d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:54.251219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5069ce0-1b64-467f-9213-12fad2b4117d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:54.270894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 281/554] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jun 20 10:51:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:54.280056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5069ce0-1b64-467f-9213-12fad2b4117d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:54.317394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:54.398477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:54.400941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5069ce0-1b64-467f-9213-12fad2b4117d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:54.406291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3cbd1-56c8-4f82-97e3-aa01c46e47fe None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:54.467514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fe59-a75b-46f4-bf64-f83eaf5d4c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:54.468246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fe59-a75b-46f4-bf64-f83eaf5d4c67 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:54.468525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fe59-a75b-46f4-bf64-f83eaf5d4c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:54.468799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227fe59-a75b-46f4-bf64-f83eaf5d4c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:54.475315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb903a-c50a-42ef-88ee-d96a0cd7d7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:54.475991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb903a-c50a-42ef-88ee-d96a0cd7d7ac None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:54.476278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb903a-c50a-42ef-88ee-d96a0cd7d7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:54.476555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb903a-c50a-42ef-88ee-d96a0cd7d7ac None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:54.477293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb903a-c50a-42ef-88ee-d96a0cd7d7ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:54.477603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb903a-c50a-42ef-88ee-d96a0cd7d7ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:54.479044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb903a-c50a-42ef-88ee-d96a0cd7d7ac None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:54.490056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227fe59-a75b-46f4-bf64-f83eaf5d4c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:54.491547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227fe59-a75b-46f4-bf64-f83eaf5d4c67 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:54.510525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 274/555] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:54.516722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb903a-c50a-42ef-88ee-d96a0cd7d7ac None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-7227fe59-a75b-46f4-bf64-f83eaf5d4c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:54.525022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:54.525967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:54.527888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:54.527888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:54.527888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:54.527888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:54.527888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 282/556] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:54.531940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fe59-a75b-46f4-bf64-f83eaf5d4c67 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:54.626394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:54.658377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227fe59-a75b-46f4-bf64-f83eaf5d4c67 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:54.695014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a171-0517-4a3c-b1b3-83750be0c21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:54.695693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a171-0517-4a3c-b1b3-83750be0c21e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:54.695978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a171-0517-4a3c-b1b3-83750be0c21e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:54.696257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54a171-0517-4a3c-b1b3-83750be0c21e None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:54.697060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54a171-0517-4a3c-b1b3-83750be0c21e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:54.697371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54a171-0517-4a3c-b1b3-83750be0c21e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:54.699038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54a171-0517-4a3c-b1b3-83750be0c21e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:54.709006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb903a-c50a-42ef-88ee-d96a0cd7d7ac None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:54.711981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 283/557] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:54.719096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a171-0517-4a3c-b1b3-83750be0c21e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:54.769387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b9b90-c054-41e3-8426-89d52476e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:54.770225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b9b90-c054-41e3-8426-89d52476e147 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:54.770509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b9b90-c054-41e3-8426-89d52476e147 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:54.770782 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b9b90-c054-41e3-8426-89d52476e147 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:54.785506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b9b90-c054-41e3-8426-89d52476e147 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:54.787029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3b9b90-c054-41e3-8426-89d52476e147 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-7d3b9b90-c054-41e3-8426-89d52476e147 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:51:54.811218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:54.812243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:54.813255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:54.813255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:54.813255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:51:54.813255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:51:54.813255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:51:54.813255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:51:54.813255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:51:54.813255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:51:54.813255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:51:54.813255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:54.821916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 275/558] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:51:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:51:54.827004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b9b90-c054-41e3-8426-89d52476e147 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:54.858495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:54.912370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54a171-0517-4a3c-b1b3-83750be0c21e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:54.965782 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:54.972345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60edd-1275-42d2-a299-68b6336c2210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:54.974243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60edd-1275-42d2-a299-68b6336c2210 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:54.974687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60edd-1275-42d2-a299-68b6336c2210 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:54.975131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60edd-1275-42d2-a299-68b6336c2210 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:54.984022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a60edd-1275-42d2-a299-68b6336c2210 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:54.986397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a60edd-1275-42d2-a299-68b6336c2210 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:54.993676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b9b90-c054-41e3-8426-89d52476e147 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-c0a60edd-1275-42d2-a299-68b6336c2210 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:51:55.006215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:55.007278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:55.008493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:51:55.008493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:51:55.008493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:51:55.008493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:51:55.008493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:51:55.008493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:51:55.008493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:55.008493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:55.011035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 284/559] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:55.012749 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141d4b8-caa7-4f02-a3bc-95e5b8f52ae3 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.013183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141d4b8-caa7-4f02-a3bc-95e5b8f52ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.013757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141d4b8-caa7-4f02-a3bc-95e5b8f52ae3 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.015787 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 285/560] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:55.017007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540760-ae74-4c30-994f-ec080ed807cb None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.017296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540760-ae74-4c30-994f-ec080ed807cb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.017578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540760-ae74-4c30-994f-ec080ed807cb None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.019087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 286/561] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:55.019999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540760-ae74-4c30-994f-ec080ed807cb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:55.039087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba38e2-1b60-40e7-9682-4bc260b1a4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:55.039770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba38e2-1b60-40e7-9682-4bc260b1a4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.040053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba38e2-1b60-40e7-9682-4bc260b1a4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.040328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba38e2-1b60-40e7-9682-4bc260b1a4a2 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.041176 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba38e2-1b60-40e7-9682-4bc260b1a4a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:55.041493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba38e2-1b60-40e7-9682-4bc260b1a4a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:55.043132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba38e2-1b60-40e7-9682-4bc260b1a4a2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:55.060719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 276/562] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jun 20 10:51:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:55.067071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b6844-e3ab-460b-af2b-68bb195f1061 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.067510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b6844-e3ab-460b-af2b-68bb195f1061 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.067919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b6844-e3ab-460b-af2b-68bb195f1061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.090704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-526b6844-e3ab-460b-af2b-68bb195f1061 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:55.091930 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-526b6844-e3ab-460b-af2b-68bb195f1061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:55.130918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540760-ae74-4c30-994f-ec080ed807cb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:55.160757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 277/563] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:55.162195 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05498d9f-2fe8-4e13-8988-3ea776a3ab9c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.162488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05498d9f-2fe8-4e13-8988-3ea776a3ab9c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.162763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05498d9f-2fe8-4e13-8988-3ea776a3ab9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.167370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:55.178233 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a705f80-0d0a-407e-8534-423c2a874bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:55.179324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a705f80-0d0a-407e-8534-423c2a874bce None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.179791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a705f80-0d0a-407e-8534-423c2a874bce None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.180145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-05498d9f-2fe8-4e13-8988-3ea776a3ab9c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:55.180462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a705f80-0d0a-407e-8534-423c2a874bce None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.181436 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a705f80-0d0a-407e-8534-423c2a874bce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:55.181602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-05498d9f-2fe8-4e13-8988-3ea776a3ab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:55.182081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a705f80-0d0a-407e-8534-423c2a874bce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:55.183846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a705f80-0d0a-407e-8534-423c2a874bce None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:55.198022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 287/564] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:55.199444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a705f80-0d0a-407e-8534-423c2a874bce None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:55.250189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 278/565] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:55.251055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05498d9f-2fe8-4e13-8988-3ea776a3ab9c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:55.253340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:55.302601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a705f80-0d0a-407e-8534-423c2a874bce None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:55.335628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41f855-8860-400c-9e8a-d8680a4c3505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:55.336279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41f855-8860-400c-9e8a-d8680a4c3505 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.336561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41f855-8860-400c-9e8a-d8680a4c3505 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.336832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41f855-8860-400c-9e8a-d8680a4c3505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.349276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05498d9f-2fe8-4e13-8988-3ea776a3ab9c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:55.380162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:55.385468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cac3f-1188-4ef9-915d-2662e72654be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:55.386387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cac3f-1188-4ef9-915d-2662e72654be None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.386745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cac3f-1188-4ef9-915d-2662e72654be None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.387063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cac3f-1188-4ef9-915d-2662e72654be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.397025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cac3f-1188-4ef9-915d-2662e72654be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:55.404310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41f855-8860-400c-9e8a-d8680a4c3505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-e71cac3f-1188-4ef9-915d-2662e72654be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:55.405151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:55.411694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:55.412753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:55.413540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41f855-8860-400c-9e8a-d8680a4c3505 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:55.413626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 279/566] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:55.413626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cac3f-1188-4ef9-915d-2662e72654be None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:55.458947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:55.507559 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 288/567] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:51:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:51:55.508838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6eb3a-e2d7-4dcc-b87d-0297ea8bf0ce None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.509151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6eb3a-e2d7-4dcc-b87d-0297ea8bf0ce None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.509432 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6eb3a-e2d7-4dcc-b87d-0297ea8bf0ce None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.510708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 289/568] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:55.511570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6eb3a-e2d7-4dcc-b87d-0297ea8bf0ce None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:55.515353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cac3f-1188-4ef9-915d-2662e72654be None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:55.550539 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382f59d-0885-478f-9a1c-07d2886da419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:55.551434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382f59d-0885-478f-9a1c-07d2886da419 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.551702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382f59d-0885-478f-9a1c-07d2886da419 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.551970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382f59d-0885-478f-9a1c-07d2886da419 None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/37b1f13aa474493d83b3a16f76949fbc/roles/2cf42fcd695a4696b148241bfd911357` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.567444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382f59d-0885-478f-9a1c-07d2886da419 None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=37b1f13aa474493d83b3a16f76949fbc, role_id=2cf42fcd695a4696b148241bfd911357)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:55.569581 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382f59d-0885-478f-9a1c-07d2886da419 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:55.583639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 280/569] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:55 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/37b1f13aa474493d83b3a16f76949fbc/roles/2cf42fcd695a4696b148241bfd911357 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:55.584522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382f59d-0885-478f-9a1c-07d2886da419 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:55.627330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6eb3a-e2d7-4dcc-b87d-0297ea8bf0ce None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:55.638249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:55.666280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0786b8-71e2-496e-b56a-c4cce5c5e67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:55.667106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0786b8-71e2-496e-b56a-c4cce5c5e67b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.667388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0786b8-71e2-496e-b56a-c4cce5c5e67b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.667892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0786b8-71e2-496e-b56a-c4cce5c5e67b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.679710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0786b8-71e2-496e-b56a-c4cce5c5e67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-8a0786b8-71e2-496e-b56a-c4cce5c5e67b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:55.687610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:55.688811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:55.689834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 290/570] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:55.691350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0786b8-71e2-496e-b56a-c4cce5c5e67b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:55.710735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:55.714387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382f59d-0885-478f-9a1c-07d2886da419 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:55.757098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7a605-f85b-4f6d-96b0-7ca5bd227317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:55.758001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a605-f85b-4f6d-96b0-7ca5bd227317 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.758324 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a605-f85b-4f6d-96b0-7ca5bd227317 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.758669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7a605-f85b-4f6d-96b0-7ca5bd227317 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.759562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7a605-f85b-4f6d-96b0-7ca5bd227317 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:55.759938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7a605-f85b-4f6d-96b0-7ca5bd227317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:55.761501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7a605-f85b-4f6d-96b0-7ca5bd227317 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:55.771188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 281/571] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:55.772522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87ed95-fb59-44ce-a114-03fc0366439a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.772887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87ed95-fb59-44ce-a114-03fc0366439a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.773234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87ed95-fb59-44ce-a114-03fc0366439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.795951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4a87ed95-fb59-44ce-a114-03fc0366439a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:55.797164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4a87ed95-fb59-44ce-a114-03fc0366439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:55.808497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0786b8-71e2-496e-b56a-c4cce5c5e67b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:55.852248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04842e5-99cb-46de-a1af-279f10a7e563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:55.853215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04842e5-99cb-46de-a1af-279f10a7e563 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:55.853597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04842e5-99cb-46de-a1af-279f10a7e563 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:55.853967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04842e5-99cb-46de-a1af-279f10a7e563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:55.878127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 282/572] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:55.878337 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04842e5-99cb-46de-a1af-279f10a7e563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:55.878429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87ed95-fb59-44ce-a114-03fc0366439a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:55.878510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04842e5-99cb-46de-a1af-279f10a7e563 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-b04842e5-99cb-46de-a1af-279f10a7e563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:55.896262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:55.898033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:55.899937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:55.899937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:55.899937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:55.899937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:55.899937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 291/573] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:55.903355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04842e5-99cb-46de-a1af-279f10a7e563 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:55.932430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:55.974588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:56.003908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87ed95-fb59-44ce-a114-03fc0366439a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.014000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04842e5-99cb-46de-a1af-279f10a7e563 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.040059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abaaa0-1abd-4668-95a6-21fef600028e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.040733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abaaa0-1abd-4668-95a6-21fef600028e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.041010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abaaa0-1abd-4668-95a6-21fef600028e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.041283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abaaa0-1abd-4668-95a6-21fef600028e None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.042033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abaaa0-1abd-4668-95a6-21fef600028e None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:56.042346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abaaa0-1abd-4668-95a6-21fef600028e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.044432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abaaa0-1abd-4668-95a6-21fef600028e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.051988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5cd47-08f7-4af8-a47a-61b18c7ca0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.051988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5cd47-08f7-4af8-a47a-61b18c7ca0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.051988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5cd47-08f7-4af8-a47a-61b18c7ca0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.051988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5cd47-08f7-4af8-a47a-61b18c7ca0e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.059549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 283/574] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:56.059549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abaaa0-1abd-4668-95a6-21fef600028e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.064508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5cd47-08f7-4af8-a47a-61b18c7ca0e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.065977 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5cd47-08f7-4af8-a47a-61b18c7ca0e6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2ee5cd47-08f7-4af8-a47a-61b18c7ca0e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:56.084459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:56.085351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:56.086249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:56.086249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:56.086249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:56.086249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:56.086249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 292/575] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:56.086525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5cd47-08f7-4af8-a47a-61b18c7ca0e6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.167533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:56.176309 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abaaa0-1abd-4668-95a6-21fef600028e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.199353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5cd47-08f7-4af8-a47a-61b18c7ca0e6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.213724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd325-1557-422f-8c49-5df9328ebfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.214857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd325-1557-422f-8c49-5df9328ebfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.215147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd325-1557-422f-8c49-5df9328ebfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.215424 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd325-1557-422f-8c49-5df9328ebfd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.216438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edd325-1557-422f-8c49-5df9328ebfd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:56.216753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edd325-1557-422f-8c49-5df9328ebfd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.217301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:56.218794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edd325-1557-422f-8c49-5df9328ebfd0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.234350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 284/576] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:56.238593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced29135-706a-4d7f-bdc5-57d974b96b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.239505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd325-1557-422f-8c49-5df9328ebfd0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.239633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced29135-706a-4d7f-bdc5-57d974b96b7e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.239942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced29135-706a-4d7f-bdc5-57d974b96b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.240169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced29135-706a-4d7f-bdc5-57d974b96b7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.252123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced29135-706a-4d7f-bdc5-57d974b96b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-ced29135-706a-4d7f-bdc5-57d974b96b7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:56.259805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:56.261263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:56.263023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:56.266813 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 293/577] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:56.267524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced29135-706a-4d7f-bdc5-57d974b96b7e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.345421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd325-1557-422f-8c49-5df9328ebfd0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.371394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced29135-706a-4d7f-bdc5-57d974b96b7e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.381854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd896a-6fb8-4d52-8ed7-3c247fcd2519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.382945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd896a-6fb8-4d52-8ed7-3c247fcd2519 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.383459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd896a-6fb8-4d52-8ed7-3c247fcd2519 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.384065 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd896a-6fb8-4d52-8ed7-3c247fcd2519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.395143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd896a-6fb8-4d52-8ed7-3c247fcd2519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.396853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd896a-6fb8-4d52-8ed7-3c247fcd2519 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.405997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac97afd-e136-4e03-8d9d-63627877192c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.406757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac97afd-e136-4e03-8d9d-63627877192c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.407040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac97afd-e136-4e03-8d9d-63627877192c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.407316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac97afd-e136-4e03-8d9d-63627877192c None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.408069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac97afd-e136-4e03-8d9d-63627877192c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:56.408379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac97afd-e136-4e03-8d9d-63627877192c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.409741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac97afd-e136-4e03-8d9d-63627877192c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-47dd896a-6fb8-4d52-8ed7-3c247fcd2519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:56.416414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:56.417971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:56.419069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:56.419069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:56.419069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:56.419069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:56.420614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 285/578] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:56.421537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd896a-6fb8-4d52-8ed7-3c247fcd2519 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.427309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 294/579] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:56.428434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac97afd-e136-4e03-8d9d-63627877192c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.433451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:56.468393 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:56.532434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd896a-6fb8-4d52-8ed7-3c247fcd2519 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.534888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac97afd-e136-4e03-8d9d-63627877192c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.567128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f639b62-0542-471d-97c0-cb59b4cc4900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.568038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f639b62-0542-471d-97c0-cb59b4cc4900 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.568411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f639b62-0542-471d-97c0-cb59b4cc4900 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.568751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f639b62-0542-471d-97c0-cb59b4cc4900 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.570742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2255e-5118-4c53-9338-7c64a35f3871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.571581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2255e-5118-4c53-9338-7c64a35f3871 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.571954 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2255e-5118-4c53-9338-7c64a35f3871 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.572322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2255e-5118-4c53-9338-7c64a35f3871 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.573176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2255e-5118-4c53-9338-7c64a35f3871 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:56.573579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2255e-5118-4c53-9338-7c64a35f3871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.575186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2255e-5118-4c53-9338-7c64a35f3871 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.576041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f639b62-0542-471d-97c0-cb59b4cc4900 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.577568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f639b62-0542-471d-97c0-cb59b4cc4900 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.583864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 295/580] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:56.589142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2255e-5118-4c53-9338-7c64a35f3871 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0f639b62-0542-471d-97c0-cb59b4cc4900 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:51:56.592518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:56.593537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:56.594741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:51:56.594741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:51:56.594741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:51:56.594741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:51:56.594741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:51:56.594741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:51:56.594741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:56.594741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:56.596065 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 286/581] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:56.596445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f639b62-0542-471d-97c0-cb59b4cc4900 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.667784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:56.704339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f639b62-0542-471d-97c0-cb59b4cc4900 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.704693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:56.710106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2255e-5118-4c53-9338-7c64a35f3871 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.740786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0b24d-b9b9-4245-a7a0-77860446a0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.741606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0b24d-b9b9-4245-a7a0-77860446a0fa None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.741885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0b24d-b9b9-4245-a7a0-77860446a0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.742204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0b24d-b9b9-4245-a7a0-77860446a0fa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.748838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27a46f-c071-4e2b-bdb6-1affbb499469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.749619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27a46f-c071-4e2b-bdb6-1affbb499469 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.750001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27a46f-c071-4e2b-bdb6-1affbb499469 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.750113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0b24d-b9b9-4245-a7a0-77860446a0fa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.750469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27a46f-c071-4e2b-bdb6-1affbb499469 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.751313 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27a46f-c071-4e2b-bdb6-1affbb499469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:56.751599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0b24d-b9b9-4245-a7a0-77860446a0fa None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.751807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27a46f-c071-4e2b-bdb6-1affbb499469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.753275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27a46f-c071-4e2b-bdb6-1affbb499469 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.766080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 296/582] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-87b0b24d-b9b9-4245-a7a0-77860446a0fa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:51:56.766756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:56.767819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:56.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:56.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:56.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:51:56.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:51:56.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:51:56.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:51:56.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:51:56.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:51:56.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 10:51:56.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:56.769578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27a46f-c071-4e2b-bdb6-1affbb499469 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.771038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 287/583] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:56.772116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0b24d-b9b9-4245-a7a0-77860446a0fa None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.872897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27a46f-c071-4e2b-bdb6-1affbb499469 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.893238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0b24d-b9b9-4245-a7a0-77860446a0fa None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:56.914816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc68c70-a39c-4057-abae-f049bbf3f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.916071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc68c70-a39c-4057-abae-f049bbf3f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.916518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc68c70-a39c-4057-abae-f049bbf3f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.916798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc68c70-a39c-4057-abae-f049bbf3f4c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.927351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc68c70-a39c-4057-abae-f049bbf3f4c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.928847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc68c70-a39c-4057-abae-f049bbf3f4c4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.929247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:56.938818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270c0de-68fb-4d96-8f9e-2a3f01a6f006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:56.939481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270c0de-68fb-4d96-8f9e-2a3f01a6f006 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:56.939762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270c0de-68fb-4d96-8f9e-2a3f01a6f006 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:56.940034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270c0de-68fb-4d96-8f9e-2a3f01a6f006 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:56.940825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270c0de-68fb-4d96-8f9e-2a3f01a6f006 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:56.941134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270c0de-68fb-4d96-8f9e-2a3f01a6f006 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:56.942610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270c0de-68fb-4d96-8f9e-2a3f01a6f006 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-bbc68c70-a39c-4057-abae-f049bbf3f4c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:56.947193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:56.948070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:56.954690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:56.954690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:56.954690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:56.954690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:56.954690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 297/584] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:56.954690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc68c70-a39c-4057-abae-f049bbf3f4c4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.964873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 288/585] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:56.964873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270c0de-68fb-4d96-8f9e-2a3f01a6f006 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:56.968881 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:57.062268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc68c70-a39c-4057-abae-f049bbf3f4c4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.068695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270c0de-68fb-4d96-8f9e-2a3f01a6f006 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.097355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40038194-764e-4b42-8fe0-05f18293eb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:57.098182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40038194-764e-4b42-8fe0-05f18293eb41 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.098472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40038194-764e-4b42-8fe0-05f18293eb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.098738 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40038194-764e-4b42-8fe0-05f18293eb41 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.104126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51a821-3b28-4802-9853-827760d07981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:57.104801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51a821-3b28-4802-9853-827760d07981 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.105207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51a821-3b28-4802-9853-827760d07981 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.105487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51a821-3b28-4802-9853-827760d07981 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.106361 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51a821-3b28-4802-9853-827760d07981 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:57.106759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51a821-3b28-4802-9853-827760d07981 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:57.106982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40038194-764e-4b42-8fe0-05f18293eb41 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:57.108233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51a821-3b28-4802-9853-827760d07981 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:57.108412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40038194-764e-4b42-8fe0-05f18293eb41 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-40038194-764e-4b42-8fe0-05f18293eb41 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:51:57.124060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:57.124999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:57.126069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 298/586] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:57.127142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40038194-764e-4b42-8fe0-05f18293eb41 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:57.129833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 289/587] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Jun 20 10:51:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:57.130981 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51a821-3b28-4802-9853-827760d07981 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:57.182507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:57.201780 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:57.234214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40038194-764e-4b42-8fe0-05f18293eb41 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.246129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51a821-3b28-4802-9853-827760d07981 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.268317 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf17622-e124-4cff-aa00-5a40b7949ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:57.268984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17622-e124-4cff-aa00-5a40b7949ece None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.269265 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17622-e124-4cff-aa00-5a40b7949ece None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.269539 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17622-e124-4cff-aa00-5a40b7949ece None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.270284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf17622-e124-4cff-aa00-5a40b7949ece None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:57.270593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf17622-e124-4cff-aa00-5a40b7949ece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:57.271956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf17622-e124-4cff-aa00-5a40b7949ece None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:57.283286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 299/588] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:57.284114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf17622-e124-4cff-aa00-5a40b7949ece None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:57.292294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ad2d6-792d-47ed-96b0-4e96619d834e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:57.293355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ad2d6-792d-47ed-96b0-4e96619d834e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.293355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ad2d6-792d-47ed-96b0-4e96619d834e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.293664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ad2d6-792d-47ed-96b0-4e96619d834e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.294509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ad2d6-792d-47ed-96b0-4e96619d834e None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=3c133005ca8d4478be8ff4acad5e88cc) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:57.294821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ad2d6-792d-47ed-96b0-4e96619d834e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:57.296273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5ad2d6-792d-47ed-96b0-4e96619d834e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:57.309095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 290/589] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:51:57 2022] GET /identity/v3/role_assignments?user.id=3c133005ca8d4478be8ff4acad5e88cc&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:57.309095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ad2d6-792d-47ed-96b0-4e96619d834e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:57.420305 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf17622-e124-4cff-aa00-5a40b7949ece None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.447694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ad2d6-792d-47ed-96b0-4e96619d834e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.460970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:57.463482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:57.466918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715d3d7-9de5-4886-8ae4-2ee23d424596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:57.467810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715d3d7-9de5-4886-8ae4-2ee23d424596 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.468096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715d3d7-9de5-4886-8ae4-2ee23d424596 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.468394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715d3d7-9de5-4886-8ae4-2ee23d424596 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.469743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715d3d7-9de5-4886-8ae4-2ee23d424596 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:57.470067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715d3d7-9de5-4886-8ae4-2ee23d424596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:57.472284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715d3d7-9de5-4886-8ae4-2ee23d424596 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:57.494490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 300/590] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jun 20 10:51:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:57.495345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715d3d7-9de5-4886-8ae4-2ee23d424596 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:57.500769 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb9d7f-b6c2-4202-b303-12b98f184401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:57.501682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb9d7f-b6c2-4202-b303-12b98f184401 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.502030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb9d7f-b6c2-4202-b303-12b98f184401 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.502405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb9d7f-b6c2-4202-b303-12b98f184401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.516074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb9d7f-b6c2-4202-b303-12b98f184401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:57.517967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb9d7f-b6c2-4202-b303-12b98f184401 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-96bb9d7f-b6c2-4202-b303-12b98f184401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:57.541188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:57.542146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:57.543430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:57.543430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:57.543430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:57.543430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:57.543430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 291/591] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:57.544465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb9d7f-b6c2-4202-b303-12b98f184401 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:57.609738 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715d3d7-9de5-4886-8ae4-2ee23d424596 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.648050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b8420-9e0e-46e0-b5f8-00d02d2a3f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:57.648966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b8420-9e0e-46e0-b5f8-00d02d2a3f02 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.649320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b8420-9e0e-46e0-b5f8-00d02d2a3f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.649520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b8420-9e0e-46e0-b5f8-00d02d2a3f02 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.657135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb9d7f-b6c2-4202-b303-12b98f184401 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.657680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b8420-9e0e-46e0-b5f8-00d02d2a3f02 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:57.659449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b8420-9e0e-46e0-b5f8-00d02d2a3f02 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-c56b8420-9e0e-46e0-b5f8-00d02d2a3f02 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:51:57.676145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:57.677116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:57.678028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 301/592] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:51:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:57.678629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b8420-9e0e-46e0-b5f8-00d02d2a3f02 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:57.695763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda0616-0b66-4f06-8f47-45c8300bf2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:57.696578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda0616-0b66-4f06-8f47-45c8300bf2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.696864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda0616-0b66-4f06-8f47-45c8300bf2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.697142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda0616-0b66-4f06-8f47-45c8300bf2d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.709540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda0616-0b66-4f06-8f47-45c8300bf2d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:57.711380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda0616-0b66-4f06-8f47-45c8300bf2d3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ceda0616-0b66-4f06-8f47-45c8300bf2d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:57.729638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:57.730899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:57.732020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:57.732020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:57.732020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:57.732020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:57.732476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:57.733875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 292/593] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:57.735189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c047e2-7939-43da-baa7-59f0eeffd66c None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.735475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c047e2-7939-43da-baa7-59f0eeffd66c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.735754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c047e2-7939-43da-baa7-59f0eeffd66c None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.738126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 293/594] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:57.738126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c047e2-7939-43da-baa7-59f0eeffd66c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:57.754443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:57.797018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b8420-9e0e-46e0-b5f8-00d02d2a3f02 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.836120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf36b29-4c18-4b04-afe0-4e75080a94b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:57.836794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36b29-4c18-4b04-afe0-4e75080a94b8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.837077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36b29-4c18-4b04-afe0-4e75080a94b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.837357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf36b29-4c18-4b04-afe0-4e75080a94b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.838081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf36b29-4c18-4b04-afe0-4e75080a94b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:57.838414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf36b29-4c18-4b04-afe0-4e75080a94b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:57.840133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf36b29-4c18-4b04-afe0-4e75080a94b8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:57.848423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c047e2-7939-43da-baa7-59f0eeffd66c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.854130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 302/595] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:57.854130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf36b29-4c18-4b04-afe0-4e75080a94b8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:57.894897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfdb3b-67ef-445f-ad44-d59afbde0fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:57.895582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfdb3b-67ef-445f-ad44-d59afbde0fbd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.895866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfdb3b-67ef-445f-ad44-d59afbde0fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.896144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfdb3b-67ef-445f-ad44-d59afbde0fbd None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.896987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfdb3b-67ef-445f-ad44-d59afbde0fbd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:57.897293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfdb3b-67ef-445f-ad44-d59afbde0fbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:57.899203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfdb3b-67ef-445f-ad44-d59afbde0fbd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:57.918408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 294/596] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jun 20 10:51:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:57.919454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23cabc-2b63-4670-b09a-d96a20bcc0cd None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:57.919745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23cabc-2b63-4670-b09a-d96a20bcc0cd None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:57.920027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23cabc-2b63-4670-b09a-d96a20bcc0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:57.936313 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ef23cabc-2b63-4670-b09a-d96a20bcc0cd None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:57.937430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ef23cabc-2b63-4670-b09a-d96a20bcc0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:57.971144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf36b29-4c18-4b04-afe0-4e75080a94b8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:57.980896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:58.006840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfea645-ea14-426f-a579-8bd90845b41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.007519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfea645-ea14-426f-a579-8bd90845b41a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.014400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfea645-ea14-426f-a579-8bd90845b41a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.014522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 295/597] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:58.014622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfea645-ea14-426f-a579-8bd90845b41a None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.014728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23cabc-2b63-4670-b09a-d96a20bcc0cd None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.014802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfea645-ea14-426f-a579-8bd90845b41a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:58.014802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfea645-ea14-426f-a579-8bd90845b41a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:58.014802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfea645-ea14-426f-a579-8bd90845b41a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:58.016323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:58.032885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 303/598] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:58.034045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfea645-ea14-426f-a579-8bd90845b41a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.120197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23cabc-2b63-4670-b09a-d96a20bcc0cd None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:58.146214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfea645-ea14-426f-a579-8bd90845b41a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:58.157302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb3c4a-ac90-4381-8748-f57ef2a36b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.158156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb3c4a-ac90-4381-8748-f57ef2a36b71 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.158439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb3c4a-ac90-4381-8748-f57ef2a36b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.158718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb3c4a-ac90-4381-8748-f57ef2a36b71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.165647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb3c4a-ac90-4381-8748-f57ef2a36b71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:58.167136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb3c4a-ac90-4381-8748-f57ef2a36b71 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-acbb3c4a-ac90-4381-8748-f57ef2a36b71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:51:58.181198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:58.182154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:58.183268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:51:58.183268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:51:58.183268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:51:58.183268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:51:58.183268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:51:58.183268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:51:58.183268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:58.183268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:58.183268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 296/599] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:58.183268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb3c4a-ac90-4381-8748-f57ef2a36b71 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.189656 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a0510-fbb3-4c04-803d-a7d5992ea507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.190428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a0510-fbb3-4c04-803d-a7d5992ea507 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.190709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a0510-fbb3-4c04-803d-a7d5992ea507 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.191005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a0510-fbb3-4c04-803d-a7d5992ea507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.201577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a0510-fbb3-4c04-803d-a7d5992ea507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:58.203068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a0510-fbb3-4c04-803d-a7d5992ea507 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1d0a0510-fbb3-4c04-803d-a7d5992ea507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:58.221064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:58.222005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:58.228069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:58.228069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:58.228069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:58.228069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:58.228069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 304/600] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:58.228069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a0510-fbb3-4c04-803d-a7d5992ea507 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.244389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:58.262763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:58.305570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb3c4a-ac90-4381-8748-f57ef2a36b71 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:58.326200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a0510-fbb3-4c04-803d-a7d5992ea507 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:58.342174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ed202-e297-48d1-8e6a-8a583cc9e8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.343131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ed202-e297-48d1-8e6a-8a583cc9e8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.343411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ed202-e297-48d1-8e6a-8a583cc9e8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.343680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ed202-e297-48d1-8e6a-8a583cc9e8d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.350687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ed202-e297-48d1-8e6a-8a583cc9e8d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:58.352154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ed202-e297-48d1-8e6a-8a583cc9e8d1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:58.365584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe179-efdd-43b4-9f84-d4e42a6b1e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.366419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe179-efdd-43b4-9f84-d4e42a6b1e5d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.366704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe179-efdd-43b4-9f84-d4e42a6b1e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.366974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe179-efdd-43b4-9f84-d4e42a6b1e5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-a33ed202-e297-48d1-8e6a-8a583cc9e8d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:51:58.367165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:58.367994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:58.368858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:51:58.368858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:51:58.368858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:51:58.368858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:51:58.368858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:51:58.368858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:51:58.368858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:58.368858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:58.368858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 297/601] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:58.369327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ed202-e297-48d1-8e6a-8a583cc9e8d1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.387396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafe179-efdd-43b4-9f84-d4e42a6b1e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-acafe179-efdd-43b4-9f84-d4e42a6b1e5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:58.395145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:58.396057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 305/602] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:58.398698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe179-efdd-43b4-9f84-d4e42a6b1e5d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.485220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ed202-e297-48d1-8e6a-8a583cc9e8d1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:58.488671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:58.509100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:58.511064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe179-efdd-43b4-9f84-d4e42a6b1e5d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:58.522444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb898a-6610-4483-9fc3-f373ae939ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.523121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb898a-6610-4483-9fc3-f373ae939ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.523409 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb898a-6610-4483-9fc3-f373ae939ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.523693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb898a-6610-4483-9fc3-f373ae939ef5 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.524496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb898a-6610-4483-9fc3-f373ae939ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:58.524812 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb898a-6610-4483-9fc3-f373ae939ef5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:58.526296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb898a-6610-4483-9fc3-f373ae939ef5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:58.538037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 298/603] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:51:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:58.538877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb898a-6610-4483-9fc3-f373ae939ef5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.553333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3418ff9-502a-4d99-a1fa-d01eed7afb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.554338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3418ff9-502a-4d99-a1fa-d01eed7afb67 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.554787 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3418ff9-502a-4d99-a1fa-d01eed7afb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.555220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3418ff9-502a-4d99-a1fa-d01eed7afb67 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.556087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3418ff9-502a-4d99-a1fa-d01eed7afb67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:58.556546 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3418ff9-502a-4d99-a1fa-d01eed7afb67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:58.558164 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3418ff9-502a-4d99-a1fa-d01eed7afb67 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:58.576748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 306/604] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:58.577844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3418ff9-502a-4d99-a1fa-d01eed7afb67 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.654397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb898a-6610-4483-9fc3-f373ae939ef5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:58.686656 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3418ff9-502a-4d99-a1fa-d01eed7afb67 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:58.690974 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f664e00-1405-46ad-8bc0-899396e9bc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.691647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f664e00-1405-46ad-8bc0-899396e9bc79 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.691932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f664e00-1405-46ad-8bc0-899396e9bc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.692209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f664e00-1405-46ad-8bc0-899396e9bc79 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.693005 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f664e00-1405-46ad-8bc0-899396e9bc79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:58.693317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f664e00-1405-46ad-8bc0-899396e9bc79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:58.694818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f664e00-1405-46ad-8bc0-899396e9bc79 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:58.713147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 299/605] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:51:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:58.713147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f664e00-1405-46ad-8bc0-899396e9bc79 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.726541 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503e802-a07f-4f8f-a0e1-8ca97307a0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.727226 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503e802-a07f-4f8f-a0e1-8ca97307a0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.727510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503e802-a07f-4f8f-a0e1-8ca97307a0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.727784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503e802-a07f-4f8f-a0e1-8ca97307a0a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.728535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503e802-a07f-4f8f-a0e1-8ca97307a0a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:58.728841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503e802-a07f-4f8f-a0e1-8ca97307a0a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:58.730340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503e802-a07f-4f8f-a0e1-8ca97307a0a5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:58.743466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 307/606] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:58.743466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503e802-a07f-4f8f-a0e1-8ca97307a0a5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.743466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:58.760155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:58.821107 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f664e00-1405-46ad-8bc0-899396e9bc79 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:58.850836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503e802-a07f-4f8f-a0e1-8ca97307a0a5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:58.860913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d0d56-f3bf-46fc-87e3-2edfe6d2c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.861925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d0d56-f3bf-46fc-87e3-2edfe6d2c073 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.862327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d0d56-f3bf-46fc-87e3-2edfe6d2c073 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.862687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d0d56-f3bf-46fc-87e3-2edfe6d2c073 None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/d73fe529c7d342f895aabcb9a39aba78/roles/2cf42fcd695a4696b148241bfd911357` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.877816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d0d56-f3bf-46fc-87e3-2edfe6d2c073 None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=d73fe529c7d342f895aabcb9a39aba78, role_id=2cf42fcd695a4696b148241bfd911357)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:58.879345 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d0d56-f3bf-46fc-87e3-2edfe6d2c073 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:58.891266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e31452-303c-4a68-903c-3cf70e94a6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:58.892064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e31452-303c-4a68-903c-3cf70e94a6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:58.892345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e31452-303c-4a68-903c-3cf70e94a6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:58.892621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e31452-303c-4a68-903c-3cf70e94a6e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:58.894618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 300/607] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:58 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/d73fe529c7d342f895aabcb9a39aba78/roles/2cf42fcd695a4696b148241bfd911357 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:58.895500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d0d56-f3bf-46fc-87e3-2edfe6d2c073 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:58.907098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e31452-303c-4a68-903c-3cf70e94a6e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:58.908925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e31452-303c-4a68-903c-3cf70e94a6e5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-89e31452-303c-4a68-903c-3cf70e94a6e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:51:58.934696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:58.935894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:58.936973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:51:58.936973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:51:58.936973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:51:58.936973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:51:58.936973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:51:58.936973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:51:58.936973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:51:58.936973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:58.942187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 308/608] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:51:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:58.942187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e31452-303c-4a68-903c-3cf70e94a6e5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:59.015116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:59.056775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:59.062649 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e31452-303c-4a68-903c-3cf70e94a6e5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:59.072964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d0d56-f3bf-46fc-87e3-2edfe6d2c073 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:59.102407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d8f45-8329-482f-a0a1-75534c5695f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:59.104070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d8f45-8329-482f-a0a1-75534c5695f8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:59.104891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d8f45-8329-482f-a0a1-75534c5695f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:59.105389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d8f45-8329-482f-a0a1-75534c5695f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:59.114971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741df62-0425-42c0-a7bd-7db5d22e6298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:59.115931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741df62-0425-42c0-a7bd-7db5d22e6298 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:59.116213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741df62-0425-42c0-a7bd-7db5d22e6298 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:59.116489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741df62-0425-42c0-a7bd-7db5d22e6298 None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/841e89a6caef4a089edc0f3b1df2fe9e/roles/2cf42fcd695a4696b148241bfd911357` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:59.119652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d8f45-8329-482f-a0a1-75534c5695f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:59.122333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d8f45-8329-482f-a0a1-75534c5695f8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:59.134674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741df62-0425-42c0-a7bd-7db5d22e6298 None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=841e89a6caef4a089edc0f3b1df2fe9e, role_id=2cf42fcd695a4696b148241bfd911357)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:59.136173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741df62-0425-42c0-a7bd-7db5d22e6298 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-203d8f45-8329-482f-a0a1-75534c5695f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:51:59.144772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:59.145647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:59.146595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:51:59.146595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:51:59.146595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:51:59.146595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:59.148875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 309/609] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:59.150034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d8f45-8329-482f-a0a1-75534c5695f8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:59.156968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 301/610] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:58 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/841e89a6caef4a089edc0f3b1df2fe9e/roles/2cf42fcd695a4696b148241bfd911357 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:59.164440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741df62-0425-42c0-a7bd-7db5d22e6298 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:59.351281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d8f45-8329-482f-a0a1-75534c5695f8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:59.371650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741df62-0425-42c0-a7bd-7db5d22e6298 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:59.380792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:59.404843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fc77e-1ed4-4235-9ca7-a971726e69ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:59.405525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc77e-1ed4-4235-9ca7-a971726e69ab None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:59.405812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc77e-1ed4-4235-9ca7-a971726e69ab None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:59.406114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc77e-1ed4-4235-9ca7-a971726e69ab None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:59.406914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fc77e-1ed4-4235-9ca7-a971726e69ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:59.407228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fc77e-1ed4-4235-9ca7-a971726e69ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:59.408673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fc77e-1ed4-4235-9ca7-a971726e69ab None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:59.413824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ebfeb-3852-4285-b97f-e11addccf419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:59.414783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebfeb-3852-4285-b97f-e11addccf419 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:59.415259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebfeb-3852-4285-b97f-e11addccf419 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:59.415751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebfeb-3852-4285-b97f-e11addccf419 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:59.416563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ebfeb-3852-4285-b97f-e11addccf419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:51:59.416873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ebfeb-3852-4285-b97f-e11addccf419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:59.419279 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ebfeb-3852-4285-b97f-e11addccf419 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:59.421157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 310/611] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:51:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:51:59.422499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c757202-6fad-4d4c-99bc-d2f6e78a443b None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:59.422797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c757202-6fad-4d4c-99bc-d2f6e78a443b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:59.423073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c757202-6fad-4d4c-99bc-d2f6e78a443b None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:59.429265 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 311/612] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:51:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:51:59.430161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c757202-6fad-4d4c-99bc-d2f6e78a443b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:59.431839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 302/613] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:51:59.435578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b43a7e-c89d-479f-bc25-c338238bfb09 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:59.435873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b43a7e-c89d-479f-bc25-c338238bfb09 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:59.436157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b43a7e-c89d-479f-bc25-c338238bfb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:59.441188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:59.462789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-33b43a7e-c89d-479f-bc25-c338238bfb09 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:51:59.464861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-33b43a7e-c89d-479f-bc25-c338238bfb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:51:59.538340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c757202-6fad-4d4c-99bc-d2f6e78a443b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:59.555607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 303/614] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:51:59.557132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b43a7e-c89d-479f-bc25-c338238bfb09 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:59.587348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df9123-cac8-4532-93b7-e90aba905361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:59.588249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df9123-cac8-4532-93b7-e90aba905361 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:59.588526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df9123-cac8-4532-93b7-e90aba905361 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:59.588795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df9123-cac8-4532-93b7-e90aba905361 None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/e4bf4d9571d541bcbba87094198dc2ff/roles/2cf42fcd695a4696b148241bfd911357` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:59.605926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df9123-cac8-4532-93b7-e90aba905361 None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=e4bf4d9571d541bcbba87094198dc2ff, role_id=2cf42fcd695a4696b148241bfd911357)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:59.607347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df9123-cac8-4532-93b7-e90aba905361 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:59.633254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 312/615] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:51:59 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/e4bf4d9571d541bcbba87094198dc2ff/roles/2cf42fcd695a4696b148241bfd911357 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:51:59.634599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df9123-cac8-4532-93b7-e90aba905361 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:59.693638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:59.740125 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b43a7e-c89d-479f-bc25-c338238bfb09 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:59.802446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:51:59.805769 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fea27-17d3-460e-88f2-3ee818ca3d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:59.806676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fea27-17d3-460e-88f2-3ee818ca3d54 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:59.806960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fea27-17d3-460e-88f2-3ee818ca3d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:59.807236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fea27-17d3-460e-88f2-3ee818ca3d54 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:59.827184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fea27-17d3-460e-88f2-3ee818ca3d54 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:51:59.828718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fea27-17d3-460e-88f2-3ee818ca3d54 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:51:59.836984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df9123-cac8-4532-93b7-e90aba905361 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-100fea27-17d3-460e-88f2-3ee818ca3d54 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:51:59.860660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:59.861609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:51:59.862575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 304/616] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jun 20 10:51:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:59.867497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fea27-17d3-460e-88f2-3ee818ca3d54 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:59.894821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bf28d-1146-4864-b1b7-aee8160fe62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:51:59.896002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bf28d-1146-4864-b1b7-aee8160fe62e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:51:59.896778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bf28d-1146-4864-b1b7-aee8160fe62e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:51:59.897704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bf28d-1146-4864-b1b7-aee8160fe62e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:51:59.917799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bf28d-1146-4864-b1b7-aee8160fe62e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e99bf28d-1146-4864-b1b7-aee8160fe62e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:51:59.928003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:51:59.928997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:51:59.929823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:51:59.935042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 313/617] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:51:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:51:59.940034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bf28d-1146-4864-b1b7-aee8160fe62e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:51:59.997156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fea27-17d3-460e-88f2-3ee818ca3d54 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:00.033284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4934ad-b3ee-4836-8b25-8fda8d3fc129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:00.034359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4934ad-b3ee-4836-8b25-8fda8d3fc129 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:00.034844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4934ad-b3ee-4836-8b25-8fda8d3fc129 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:00.035315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4934ad-b3ee-4836-8b25-8fda8d3fc129 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:00.036469 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4934ad-b3ee-4836-8b25-8fda8d3fc129 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:00.037025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4934ad-b3ee-4836-8b25-8fda8d3fc129 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:00.038703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4934ad-b3ee-4836-8b25-8fda8d3fc129 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:00.057834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 305/618] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jun 20 10:51:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:00.063267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4934ad-b3ee-4836-8b25-8fda8d3fc129 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:00.090049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:00.111142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bf28d-1146-4864-b1b7-aee8160fe62e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:00.152959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5144108-e4be-4afc-8031-650b68945d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:00.154423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5144108-e4be-4afc-8031-650b68945d1b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:00.154932 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5144108-e4be-4afc-8031-650b68945d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:00.155420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5144108-e4be-4afc-8031-650b68945d1b None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:00.156394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5144108-e4be-4afc-8031-650b68945d1b None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:00.156924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5144108-e4be-4afc-8031-650b68945d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:00.158601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5144108-e4be-4afc-8031-650b68945d1b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:00.169956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 314/619] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:51:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:00.175212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5144108-e4be-4afc-8031-650b68945d1b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:00.195633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:00.294784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4934ad-b3ee-4836-8b25-8fda8d3fc129 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:00.364479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5144108-e4be-4afc-8031-650b68945d1b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:00.374756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e448d6b-7093-47e2-82b6-d7f158a79267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:00.378108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e448d6b-7093-47e2-82b6-d7f158a79267 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:00.378108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e448d6b-7093-47e2-82b6-d7f158a79267 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:00.378108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e448d6b-7093-47e2-82b6-d7f158a79267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:00.438304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433a862-ac36-4590-8271-7eae1d9285f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:00.439112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433a862-ac36-4590-8271-7eae1d9285f4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:00.439388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433a862-ac36-4590-8271-7eae1d9285f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:00.440704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433a862-ac36-4590-8271-7eae1d9285f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:00.465659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433a862-ac36-4590-8271-7eae1d9285f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:00.467183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433a862-ac36-4590-8271-7eae1d9285f4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:00.494484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6433a862-ac36-4590-8271-7eae1d9285f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:00.505167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:00.506154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:00.507467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:00.507467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:00.507467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:00.507467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:00.516357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e448d6b-7093-47e2-82b6-d7f158a79267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:00.517851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e448d6b-7093-47e2-82b6-d7f158a79267 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:00.518165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 315/620] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:00.526851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433a862-ac36-4590-8271-7eae1d9285f4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:00.629518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:00.763016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 306/621] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:52:00.763670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433a862-ac36-4590-8271-7eae1d9285f4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:00.766608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e448d6b-7093-47e2-82b6-d7f158a79267 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:00.872763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a27d5-2c8a-4169-b7d0-d4654d8b7d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:00.873459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a27d5-2c8a-4169-b7d0-d4654d8b7d86 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:00.873742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a27d5-2c8a-4169-b7d0-d4654d8b7d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:00.874016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a27d5-2c8a-4169-b7d0-d4654d8b7d86 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:00.874848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a27d5-2c8a-4169-b7d0-d4654d8b7d86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:00.877842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a27d5-2c8a-4169-b7d0-d4654d8b7d86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:00.877842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a27d5-2c8a-4169-b7d0-d4654d8b7d86 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:00.921231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 316/622] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:00.928613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a27d5-2c8a-4169-b7d0-d4654d8b7d86 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:01.047737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e448d6b-7093-47e2-82b6-d7f158a79267 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:01.102457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:01.123260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54564794-8ddc-4405-bafa-b64caf3b11c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:01.124161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564794-8ddc-4405-bafa-b64caf3b11c0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.124446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564794-8ddc-4405-bafa-b64caf3b11c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.124723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54564794-8ddc-4405-bafa-b64caf3b11c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.149836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54564794-8ddc-4405-bafa-b64caf3b11c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:01.151432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54564794-8ddc-4405-bafa-b64caf3b11c0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:01.174561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a27d5-2c8a-4169-b7d0-d4654d8b7d86 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-54564794-8ddc-4405-bafa-b64caf3b11c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:01.184484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:01.185367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:01.186280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:01.186280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:01.186280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:01.186280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:01.196116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 307/623] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:01.197037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54564794-8ddc-4405-bafa-b64caf3b11c0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:01.202397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:01.248687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfeffdf-292b-40de-9b88-4502a47c83a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:01.249512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfeffdf-292b-40de-9b88-4502a47c83a9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.249797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfeffdf-292b-40de-9b88-4502a47c83a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.250073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfeffdf-292b-40de-9b88-4502a47c83a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.258633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfeffdf-292b-40de-9b88-4502a47c83a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:01.260107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfeffdf-292b-40de-9b88-4502a47c83a9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-0cfeffdf-292b-40de-9b88-4502a47c83a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:01.275930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:01.276984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:01.278038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 317/624] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:01.283139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14cc71-a152-4a70-a43e-523e273f4154 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.283434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14cc71-a152-4a70-a43e-523e273f4154 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.283717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14cc71-a152-4a70-a43e-523e273f4154 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.285495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 318/625] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:01.286984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14cc71-a152-4a70-a43e-523e273f4154 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:01.337567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54564794-8ddc-4405-bafa-b64caf3b11c0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:01.374539 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ab6ee-c306-4eb9-b23e-d61ed08fe0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:01.375299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ab6ee-c306-4eb9-b23e-d61ed08fe0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.375586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ab6ee-c306-4eb9-b23e-d61ed08fe0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.375963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ab6ee-c306-4eb9-b23e-d61ed08fe0c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.376694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ab6ee-c306-4eb9-b23e-d61ed08fe0c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:01.376999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ab6ee-c306-4eb9-b23e-d61ed08fe0c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:01.378553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ab6ee-c306-4eb9-b23e-d61ed08fe0c9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:01.392261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 308/626] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:01.393575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5265e-2ea7-4273-803b-083559a3f7e8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.393910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5265e-2ea7-4273-803b-083559a3f7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.394208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5265e-2ea7-4273-803b-083559a3f7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.395456 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14cc71-a152-4a70-a43e-523e273f4154 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:01.411886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-74e5265e-2ea7-4273-803b-083559a3f7e8 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:01.413068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-74e5265e-2ea7-4273-803b-083559a3f7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:01.414564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:01.433385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbea2e-5705-4f47-8985-b5c01d19513b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:01.433955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbea2e-5705-4f47-8985-b5c01d19513b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.434283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbea2e-5705-4f47-8985-b5c01d19513b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.434538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbea2e-5705-4f47-8985-b5c01d19513b None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.435491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbea2e-5705-4f47-8985-b5c01d19513b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:01.435807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbea2e-5705-4f47-8985-b5c01d19513b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:01.437246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbea2e-5705-4f47-8985-b5c01d19513b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:01.461067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 319/627] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jun 20 10:52:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:01.461985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbea2e-5705-4f47-8985-b5c01d19513b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:01.479459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:01.491795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 309/628] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:52:01.492556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5265e-2ea7-4273-803b-083559a3f7e8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:01.582781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbea2e-5705-4f47-8985-b5c01d19513b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:01.602421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5265e-2ea7-4273-803b-083559a3f7e8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:01.619383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea1c8b-2212-4d10-9da2-b94a054fc7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:01.621135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea1c8b-2212-4d10-9da2-b94a054fc7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.621135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea1c8b-2212-4d10-9da2-b94a054fc7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.621376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea1c8b-2212-4d10-9da2-b94a054fc7d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.628188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea1c8b-2212-4d10-9da2-b94a054fc7d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:01.629633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea1c8b-2212-4d10-9da2-b94a054fc7d1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:01.643049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e2c19-9a61-44b8-8f16-6a86403fdeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a5ea1c8b-2212-4d10-9da2-b94a054fc7d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:01.643540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:01.644391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:01.645283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:01.645283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:01.645283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:01.645283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:01.645283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:01.645283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:01.645283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:01.645283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:01.645721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2c19-9a61-44b8-8f16-6a86403fdeda None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.645721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2c19-9a61-44b8-8f16-6a86403fdeda None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.645721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2c19-9a61-44b8-8f16-6a86403fdeda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.649510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 320/629] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:01.650887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afa70c-5228-45df-b828-7457b673379f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.651176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afa70c-5228-45df-b828-7457b673379f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.651455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afa70c-5228-45df-b828-7457b673379f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.657479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 321/630] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:01.657479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afa70c-5228-45df-b828-7457b673379f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:01.662050 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e2c19-9a61-44b8-8f16-6a86403fdeda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:01.663981 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e2c19-9a61-44b8-8f16-6a86403fdeda None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f97e2c19-9a61-44b8-8f16-6a86403fdeda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:01.684805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:01.685921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:01.687223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:01.687223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:01.687223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:01.687223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:01.690518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 310/631] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:01.691472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e2c19-9a61-44b8-8f16-6a86403fdeda None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:01.693992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:01.743874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:01.762554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afa70c-5228-45df-b828-7457b673379f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:01.800772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8ce10-67c2-4227-af77-1712ae985259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:01.801688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8ce10-67c2-4227-af77-1712ae985259 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.802037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8ce10-67c2-4227-af77-1712ae985259 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.802449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8ce10-67c2-4227-af77-1712ae985259 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.807397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e2c19-9a61-44b8-8f16-6a86403fdeda None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:01.813698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc8ce10-67c2-4227-af77-1712ae985259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6fc8ce10-67c2-4227-af77-1712ae985259 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:01.821483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:01.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:01.823897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:01.824945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 322/632] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:52:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:01.826115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d4869-1064-4616-9897-cc669c0a0134 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.826471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d4869-1064-4616-9897-cc669c0a0134 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.826826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d4869-1064-4616-9897-cc669c0a0134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.846572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c61d4869-1064-4616-9897-cc669c0a0134 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:01.847860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c61d4869-1064-4616-9897-cc669c0a0134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:01.852162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf631e5-92ff-411c-9e55-9d1bcf3bf2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:01.852908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf631e5-92ff-411c-9e55-9d1bcf3bf2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:01.853257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf631e5-92ff-411c-9e55-9d1bcf3bf2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:01.853591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf631e5-92ff-411c-9e55-9d1bcf3bf2e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:01.854491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf631e5-92ff-411c-9e55-9d1bcf3bf2e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:01.855081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf631e5-92ff-411c-9e55-9d1bcf3bf2e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:01.856560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf631e5-92ff-411c-9e55-9d1bcf3bf2e1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:01.874312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 311/633] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:01.874312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf631e5-92ff-411c-9e55-9d1bcf3bf2e1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:01.932442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 323/634] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:52:01.933893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d4869-1064-4616-9897-cc669c0a0134 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:01.968620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:01.995879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf631e5-92ff-411c-9e55-9d1bcf3bf2e1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:01.999182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:02.033729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398a5fa-63d7-40f9-a479-6183deb9188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.034426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398a5fa-63d7-40f9-a479-6183deb9188e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.034710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398a5fa-63d7-40f9-a479-6183deb9188e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.035128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398a5fa-63d7-40f9-a479-6183deb9188e None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.035838 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398a5fa-63d7-40f9-a479-6183deb9188e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:02.036245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398a5fa-63d7-40f9-a479-6183deb9188e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:02.037668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398a5fa-63d7-40f9-a479-6183deb9188e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:02.046973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d4869-1064-4616-9897-cc669c0a0134 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:02.056019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 312/635] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:02.056019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398a5fa-63d7-40f9-a479-6183deb9188e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.081561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339da8cc-6a28-4d20-a00d-67f0a68c2a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.082270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339da8cc-6a28-4d20-a00d-67f0a68c2a8e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.082913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339da8cc-6a28-4d20-a00d-67f0a68c2a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.083184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339da8cc-6a28-4d20-a00d-67f0a68c2a8e None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.083945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339da8cc-6a28-4d20-a00d-67f0a68c2a8e None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:02.084221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339da8cc-6a28-4d20-a00d-67f0a68c2a8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:02.085587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339da8cc-6a28-4d20-a00d-67f0a68c2a8e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:02.093441 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 324/636] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:52:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:02.094239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339da8cc-6a28-4d20-a00d-67f0a68c2a8e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.156624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398a5fa-63d7-40f9-a479-6183deb9188e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:02.176604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:02.189961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339da8cc-6a28-4d20-a00d-67f0a68c2a8e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:02.190514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117963d2-d9b1-43fc-b988-547645cfe986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.191170 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117963d2-d9b1-43fc-b988-547645cfe986 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.191454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117963d2-d9b1-43fc-b988-547645cfe986 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.191710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117963d2-d9b1-43fc-b988-547645cfe986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.192539 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117963d2-d9b1-43fc-b988-547645cfe986 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=37b1f13aa474493d83b3a16f76949fbc) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:02.192851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117963d2-d9b1-43fc-b988-547645cfe986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:02.194318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117963d2-d9b1-43fc-b988-547645cfe986 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:02.200297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 313/637] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:02 2022] GET /identity/v3/role_assignments?user.id=37b1f13aa474493d83b3a16f76949fbc&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:02.201083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117963d2-d9b1-43fc-b988-547645cfe986 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.233272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a32201-4c06-4d47-abec-04dc1567c140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.233885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:02.233994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32201-4c06-4d47-abec-04dc1567c140 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.234208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32201-4c06-4d47-abec-04dc1567c140 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.234484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32201-4c06-4d47-abec-04dc1567c140 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.247350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a32201-4c06-4d47-abec-04dc1567c140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-45a32201-4c06-4d47-abec-04dc1567c140 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:02.255269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:02.256327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:02.257159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 325/638] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:52:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:02.257965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bb70a-b44c-4089-8dd3-5fff02466dbb None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.259175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bb70a-b44c-4089-8dd3-5fff02466dbb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.259175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bb70a-b44c-4089-8dd3-5fff02466dbb None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.260174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 326/639] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:02.260799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bb70a-b44c-4089-8dd3-5fff02466dbb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.331360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117963d2-d9b1-43fc-b988-547645cfe986 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:02.373779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d835b-c318-4641-bb89-ecc6b700c304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.374628 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d835b-c318-4641-bb89-ecc6b700c304 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.374908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d835b-c318-4641-bb89-ecc6b700c304 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.375183 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d835b-c318-4641-bb89-ecc6b700c304 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.378074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bb70a-b44c-4089-8dd3-5fff02466dbb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:02.382647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d835b-c318-4641-bb89-ecc6b700c304 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:02.384113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d835b-c318-4641-bb89-ecc6b700c304 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ba8d835b-c318-4641-bb89-ecc6b700c304 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:02.398898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:02.399982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:02.400900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:02.400900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:02.400900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:02.400900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:02.400900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:02.400900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:02.400900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:02.400900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:02.400900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 314/640] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:02.401372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d835b-c318-4641-bb89-ecc6b700c304 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.420019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74611141-bcb8-4a3e-b10e-f554204b1d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.420857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611141-bcb8-4a3e-b10e-f554204b1d8a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.421416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611141-bcb8-4a3e-b10e-f554204b1d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.421416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611141-bcb8-4a3e-b10e-f554204b1d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.436036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74611141-bcb8-4a3e-b10e-f554204b1d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:02.437829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74611141-bcb8-4a3e-b10e-f554204b1d8a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:02.446819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-74611141-bcb8-4a3e-b10e-f554204b1d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:02.458245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:02.459243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:02.460148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:02.460148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:02.460148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:02.460148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:02.460367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 327/641] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:02.461445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88b92d-5208-4446-80b3-829ecf792101 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.461730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88b92d-5208-4446-80b3-829ecf792101 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.462003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88b92d-5208-4446-80b3-829ecf792101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.480245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ac88b92d-5208-4446-80b3-829ecf792101 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:02.481542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ac88b92d-5208-4446-80b3-829ecf792101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:02.501024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:02.514790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d835b-c318-4641-bb89-ecc6b700c304 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:02.550586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a8b4d-a529-428b-87f7-6f2cbbac0952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.551290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a8b4d-a529-428b-87f7-6f2cbbac0952 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.551625 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a8b4d-a529-428b-87f7-6f2cbbac0952 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.551845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a8b4d-a529-428b-87f7-6f2cbbac0952 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.552591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a8b4d-a529-428b-87f7-6f2cbbac0952 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:02.552896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a8b4d-a529-428b-87f7-6f2cbbac0952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:02.553229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 328/642] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:52:02.554115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88b92d-5208-4446-80b3-829ecf792101 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.559668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a8b4d-a529-428b-87f7-6f2cbbac0952 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:02.576426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 315/643] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:52:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:02.576426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a8b4d-a529-428b-87f7-6f2cbbac0952 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.663714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88b92d-5208-4446-80b3-829ecf792101 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:02.679693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a8b4d-a529-428b-87f7-6f2cbbac0952 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:02.696423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:02.703417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b827c0b-7481-4fe8-bdc5-7d08b1b7e2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.704090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b827c0b-7481-4fe8-bdc5-7d08b1b7e2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.704373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b827c0b-7481-4fe8-bdc5-7d08b1b7e2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.704711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b827c0b-7481-4fe8-bdc5-7d08b1b7e2e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.705451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b827c0b-7481-4fe8-bdc5-7d08b1b7e2e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:02.705764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b827c0b-7481-4fe8-bdc5-7d08b1b7e2e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:02.707272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b827c0b-7481-4fe8-bdc5-7d08b1b7e2e3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:02.718335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3246d85-2c67-4406-b6f2-d21b10cc5217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.719008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3246d85-2c67-4406-b6f2-d21b10cc5217 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.719294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3246d85-2c67-4406-b6f2-d21b10cc5217 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.719573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3246d85-2c67-4406-b6f2-d21b10cc5217 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.720253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 329/644] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:02.720901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b827c0b-7481-4fe8-bdc5-7d08b1b7e2e3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.724965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3246d85-2c67-4406-b6f2-d21b10cc5217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:02.725290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3246d85-2c67-4406-b6f2-d21b10cc5217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:02.726750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3246d85-2c67-4406-b6f2-d21b10cc5217 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:02.743688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 316/645] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:02.743688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3246d85-2c67-4406-b6f2-d21b10cc5217 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.754346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:02.834624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b827c0b-7481-4fe8-bdc5-7d08b1b7e2e3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:02.850141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3246d85-2c67-4406-b6f2-d21b10cc5217 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:02.869687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ffa8-982c-40a7-b7a8-61441af3e485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.870831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ffa8-982c-40a7-b7a8-61441af3e485 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.871224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ffa8-982c-40a7-b7a8-61441af3e485 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.871601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4ffa8-982c-40a7-b7a8-61441af3e485 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.882951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4ffa8-982c-40a7-b7a8-61441af3e485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:02.885015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff90387-be48-41ca-adaa-00cf955d1381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:02.885922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff90387-be48-41ca-adaa-00cf955d1381 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:02.886487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff90387-be48-41ca-adaa-00cf955d1381 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:02.887060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff90387-be48-41ca-adaa-00cf955d1381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-4eb4ffa8-982c-40a7-b7a8-61441af3e485 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:02.890929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:02.892045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 330/646] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:52:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:02.898076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ffa8-982c-40a7-b7a8-61441af3e485 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.907026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff90387-be48-41ca-adaa-00cf955d1381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:02.908851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff90387-be48-41ca-adaa-00cf955d1381 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-9ff90387-be48-41ca-adaa-00cf955d1381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:02.931948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:02.933257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:02.939555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:02.939555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:02.939555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:02.939555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:02.939555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 317/647] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:02.939555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff90387-be48-41ca-adaa-00cf955d1381 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:02.955826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:03.003077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:03.011743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4ffa8-982c-40a7-b7a8-61441af3e485 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:03.047946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff90387-be48-41ca-adaa-00cf955d1381 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:03.052007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88469709-1b97-4ebc-a71a-c620194c91ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:03.053073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88469709-1b97-4ebc-a71a-c620194c91ad None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:03.053450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88469709-1b97-4ebc-a71a-c620194c91ad None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:03.053847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88469709-1b97-4ebc-a71a-c620194c91ad None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/3c133005ca8d4478be8ff4acad5e88cc/roles/2cf42fcd695a4696b148241bfd911357` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:03.068840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88469709-1b97-4ebc-a71a-c620194c91ad None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=3c133005ca8d4478be8ff4acad5e88cc, role_id=2cf42fcd695a4696b148241bfd911357)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:03.070514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88469709-1b97-4ebc-a71a-c620194c91ad None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:03.084270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 331/648] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:52:02 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/3c133005ca8d4478be8ff4acad5e88cc/roles/2cf42fcd695a4696b148241bfd911357 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:52:03.085199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88469709-1b97-4ebc-a71a-c620194c91ad None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:03.095091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf079a9c-f701-4075-ad74-f03043a9c7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:03.096121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079a9c-f701-4075-ad74-f03043a9c7ea None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:03.096502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079a9c-f701-4075-ad74-f03043a9c7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:03.096904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079a9c-f701-4075-ad74-f03043a9c7ea None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:03.105806 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf079a9c-f701-4075-ad74-f03043a9c7ea None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:03.107495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf079a9c-f701-4075-ad74-f03043a9c7ea None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-bf079a9c-f701-4075-ad74-f03043a9c7ea None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:03.124334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:03.125638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:03.126861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:03.126861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:03.126861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:03.126861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:03.126861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:03.126861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:03.126861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:03.126861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:03.126861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 10:52:03.126861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:03.128749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 318/649] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:52:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:03.129778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf079a9c-f701-4075-ad74-f03043a9c7ea None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:03.237189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:03.241095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88469709-1b97-4ebc-a71a-c620194c91ad None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:03.242112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf079a9c-f701-4075-ad74-f03043a9c7ea None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:03.271249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:03.287842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd589c0a-72b3-41d0-af12-71247d2acf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:03.288529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd589c0a-72b3-41d0-af12-71247d2acf42 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:03.288818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd589c0a-72b3-41d0-af12-71247d2acf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:03.289098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd589c0a-72b3-41d0-af12-71247d2acf42 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:03.289842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd589c0a-72b3-41d0-af12-71247d2acf42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:03.290233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd589c0a-72b3-41d0-af12-71247d2acf42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:03.290474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ab334-ae80-471a-bfec-a9a1d4d632b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:03.291312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ab334-ae80-471a-bfec-a9a1d4d632b7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:03.291635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd589c0a-72b3-41d0-af12-71247d2acf42 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:03.291955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ab334-ae80-471a-bfec-a9a1d4d632b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:03.292539 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ab334-ae80-471a-bfec-a9a1d4d632b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:03.293435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ab334-ae80-471a-bfec-a9a1d4d632b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:03.293892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ab334-ae80-471a-bfec-a9a1d4d632b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:03.295445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ab334-ae80-471a-bfec-a9a1d4d632b7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:03.304600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 332/650] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:52:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:03.305230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 319/651] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:03.305862 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ab334-ae80-471a-bfec-a9a1d4d632b7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:03.310803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd589c0a-72b3-41d0-af12-71247d2acf42 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:03.434072 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ab334-ae80-471a-bfec-a9a1d4d632b7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:03.441898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd589c0a-72b3-41d0-af12-71247d2acf42 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:03.478079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34bde9-2650-46e3-9c83-3a7b0a24c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:03.479088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34bde9-2650-46e3-9c83-3a7b0a24c4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:03.479441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34bde9-2650-46e3-9c83-3a7b0a24c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:03.479809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34bde9-2650-46e3-9c83-3a7b0a24c4d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:03.481385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bce931-7aa0-4f45-96ab-c1137d52ab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:03.482535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bce931-7aa0-4f45-96ab-c1137d52ab49 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:03.483389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bce931-7aa0-4f45-96ab-c1137d52ab49 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:03.483822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bce931-7aa0-4f45-96ab-c1137d52ab49 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:03.484820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bce931-7aa0-4f45-96ab-c1137d52ab49 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:03.485276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bce931-7aa0-4f45-96ab-c1137d52ab49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:03.486916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bce931-7aa0-4f45-96ab-c1137d52ab49 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:03.492411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34bde9-2650-46e3-9c83-3a7b0a24c4d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:03.493954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34bde9-2650-46e3-9c83-3a7b0a24c4d5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:03.509628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 333/652] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Jun 20 10:52:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:03.510721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bce931-7aa0-4f45-96ab-c1137d52ab49 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0b34bde9-2650-46e3-9c83-3a7b0a24c4d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:03.517682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:03.518790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:03.520013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:03.520013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:03.520013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:03.520013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:03.525583 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 320/653] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:03.526486 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34bde9-2650-46e3-9c83-3a7b0a24c4d5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:03.539188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:03.563436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:03.641353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bce931-7aa0-4f45-96ab-c1137d52ab49 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:03.643522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34bde9-2650-46e3-9c83-3a7b0a24c4d5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:03.677816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cb87a-42e8-4e6f-9819-39864960ac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:03.678661 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cb87a-42e8-4e6f-9819-39864960ac64 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:03.678937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cb87a-42e8-4e6f-9819-39864960ac64 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:03.679204 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cb87a-42e8-4e6f-9819-39864960ac64 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:03.686587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cb87a-42e8-4e6f-9819-39864960ac64 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:03.687239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d5d34-44b2-4a84-865d-dec7d4ea89f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:03.687973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d5d34-44b2-4a84-865d-dec7d4ea89f9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:03.688102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cb87a-42e8-4e6f-9819-39864960ac64 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:03.688447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d5d34-44b2-4a84-865d-dec7d4ea89f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:03.688872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d5d34-44b2-4a84-865d-dec7d4ea89f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-205cb87a-42e8-4e6f-9819-39864960ac64 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:03.704888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:03.705903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:03.706895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 334/654] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:03.708569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cb87a-42e8-4e6f-9819-39864960ac64 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:03.771513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d5d34-44b2-4a84-865d-dec7d4ea89f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:03.772979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d5d34-44b2-4a84-865d-dec7d4ea89f9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:03.794923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:03.806739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:03.824248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cb87a-42e8-4e6f-9819-39864960ac64 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:03.863975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6957af3-7c8d-48e0-817a-f3d936610575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:03.864646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6957af3-7c8d-48e0-817a-f3d936610575 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:03.864925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6957af3-7c8d-48e0-817a-f3d936610575 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:03.865201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6957af3-7c8d-48e0-817a-f3d936610575 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:03.865926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6957af3-7c8d-48e0-817a-f3d936610575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:03.866266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6957af3-7c8d-48e0-817a-f3d936610575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:03.867645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6957af3-7c8d-48e0-817a-f3d936610575 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:03.880665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 335/655] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:03.881627 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6957af3-7c8d-48e0-817a-f3d936610575 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:03.889443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 321/656] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:52:03.890349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d5d34-44b2-4a84-865d-dec7d4ea89f9 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:03.995703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6957af3-7c8d-48e0-817a-f3d936610575 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:03.999215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d5d34-44b2-4a84-865d-dec7d4ea89f9 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:04.032642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541bf3d-83dc-4efa-afa4-d8817fa2a0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:04.033388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541bf3d-83dc-4efa-afa4-d8817fa2a0ef None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.033668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541bf3d-83dc-4efa-afa4-d8817fa2a0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.033949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541bf3d-83dc-4efa-afa4-d8817fa2a0ef None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.034874 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541bf3d-83dc-4efa-afa4-d8817fa2a0ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:04.035145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541bf3d-83dc-4efa-afa4-d8817fa2a0ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:04.035597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c83599-149d-464d-a117-0d3d1d5e0d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:04.036410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c83599-149d-464d-a117-0d3d1d5e0d9d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.036577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541bf3d-83dc-4efa-afa4-d8817fa2a0ef None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:04.036725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c83599-149d-464d-a117-0d3d1d5e0d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.036959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c83599-149d-464d-a117-0d3d1d5e0d9d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.045229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c83599-149d-464d-a117-0d3d1d5e0d9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:04.046686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c83599-149d-464d-a117-0d3d1d5e0d9d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:04.046971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:04.048328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:04.056442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 336/657] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jun 20 10:52:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:04.057386 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541bf3d-83dc-4efa-afa4-d8817fa2a0ef None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-05c83599-149d-464d-a117-0d3d1d5e0d9d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:04.069798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:04.072093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 322/658] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:04.074923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c83599-149d-464d-a117-0d3d1d5e0d9d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:04.164872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541bf3d-83dc-4efa-afa4-d8817fa2a0ef None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:04.182158 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c83599-149d-464d-a117-0d3d1d5e0d9d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:04.201224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e8258-4e12-433c-bd73-e44f28ff5355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:04.202033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e8258-4e12-433c-bd73-e44f28ff5355 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.202360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e8258-4e12-433c-bd73-e44f28ff5355 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.202642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e8258-4e12-433c-bd73-e44f28ff5355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.213672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e8258-4e12-433c-bd73-e44f28ff5355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:04.215556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e8258-4e12-433c-bd73-e44f28ff5355 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:04.221663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68aff6e-44c0-475a-b28d-724c126248ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:04.222536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68aff6e-44c0-475a-b28d-724c126248ec None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.223012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68aff6e-44c0-475a-b28d-724c126248ec None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.223404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68aff6e-44c0-475a-b28d-724c126248ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-cf0e8258-4e12-433c-bd73-e44f28ff5355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:04.234548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:04.235866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:04.236966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:04.236966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:04.236966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:04.236966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:04.236966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 337/659] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:04.236966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e8258-4e12-433c-bd73-e44f28ff5355 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:04.288963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:04.296598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:04.306002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68aff6e-44c0-475a-b28d-724c126248ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:04.307715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68aff6e-44c0-475a-b28d-724c126248ec None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:04.342483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e8258-4e12-433c-bd73-e44f28ff5355 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:04.384720 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e394830-7f86-4828-9d9d-4f269086faa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:04.385702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e394830-7f86-4828-9d9d-4f269086faa9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.386149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e394830-7f86-4828-9d9d-4f269086faa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.386530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e394830-7f86-4828-9d9d-4f269086faa9 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.387601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e394830-7f86-4828-9d9d-4f269086faa9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:04.388015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e394830-7f86-4828-9d9d-4f269086faa9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:04.390129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e394830-7f86-4828-9d9d-4f269086faa9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:04.410736 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 338/660] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Jun 20 10:52:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:04.411445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e394830-7f86-4828-9d9d-4f269086faa9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:04.424190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 323/661] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:52:04.425903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53da0-53fe-4483-b30b-2a8d9f87e4d8 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.425903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53da0-53fe-4483-b30b-2a8d9f87e4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.425903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53da0-53fe-4483-b30b-2a8d9f87e4d8 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.427523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 324/662] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:04.428236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be53da0-53fe-4483-b30b-2a8d9f87e4d8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:04.526273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e394830-7f86-4828-9d9d-4f269086faa9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:04.536942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:04.538374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:04.548775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be53da0-53fe-4483-b30b-2a8d9f87e4d8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:04.571589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60668966-9ade-4b7e-b2f6-d2cda333f839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:04.572266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668966-9ade-4b7e-b2f6-d2cda333f839 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.572550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668966-9ade-4b7e-b2f6-d2cda333f839 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.572829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60668966-9ade-4b7e-b2f6-d2cda333f839 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.573725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60668966-9ade-4b7e-b2f6-d2cda333f839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:04.574040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60668966-9ade-4b7e-b2f6-d2cda333f839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:04.575489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60668966-9ade-4b7e-b2f6-d2cda333f839 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:04.589360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 339/663] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:04.590402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60668966-9ade-4b7e-b2f6-d2cda333f839 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:04.595765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141f89b-3863-4756-946e-ba989d556070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:04.596434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141f89b-3863-4756-946e-ba989d556070 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.596721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141f89b-3863-4756-946e-ba989d556070 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.596977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141f89b-3863-4756-946e-ba989d556070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.679646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141f89b-3863-4756-946e-ba989d556070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:04.681135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141f89b-3863-4756-946e-ba989d556070 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:04.725144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60668966-9ade-4b7e-b2f6-d2cda333f839 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:04.762783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a60e1e-fab7-4487-80bd-d4c8ae505e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:04.763681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a60e1e-fab7-4487-80bd-d4c8ae505e73 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.764027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a60e1e-fab7-4487-80bd-d4c8ae505e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.764364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a60e1e-fab7-4487-80bd-d4c8ae505e73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.777872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a60e1e-fab7-4487-80bd-d4c8ae505e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:04.779448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a60e1e-fab7-4487-80bd-d4c8ae505e73 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:04.790400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-67a60e1e-fab7-4487-80bd-d4c8ae505e73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:04.797690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:04.798859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:04.799968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:04.799968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:04.799968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:04.799968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:04.804906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 340/664] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:04.804906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3800-b2bf-4177-ae98-112c067f885c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.804906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3800-b2bf-4177-ae98-112c067f885c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.804906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3800-b2bf-4177-ae98-112c067f885c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.808373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 325/665] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:52:04.814995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141f89b-3863-4756-946e-ba989d556070 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:04.828424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ed3f3800-b2bf-4177-ae98-112c067f885c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:04.829770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ed3f3800-b2bf-4177-ae98-112c067f885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:04.846603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:04.908182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 341/666] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:52:04.909156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f3800-b2bf-4177-ae98-112c067f885c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:04.938336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141f89b-3863-4756-946e-ba989d556070 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:04.974938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dec0ae-50a3-4c9e-8924-d7b2238fc57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:04.975994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dec0ae-50a3-4c9e-8924-d7b2238fc57d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:04.976339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dec0ae-50a3-4c9e-8924-d7b2238fc57d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:04.976678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dec0ae-50a3-4c9e-8924-d7b2238fc57d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:04.983743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dec0ae-50a3-4c9e-8924-d7b2238fc57d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:04.985280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dec0ae-50a3-4c9e-8924-d7b2238fc57d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-b0dec0ae-50a3-4c9e-8924-d7b2238fc57d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:04.999202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:05.000346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:05.001317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:05.001317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:05.001317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:05.001317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:05.001317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:05.001317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:05.001317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:05.001317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:05.001317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 326/667] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:05.002604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dec0ae-50a3-4c9e-8924-d7b2238fc57d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.029288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f3800-b2bf-4177-ae98-112c067f885c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.062205 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:05.066913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d6641-e357-46ed-b2f4-c1b5c51fb6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:05.067534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d6641-e357-46ed-b2f4-c1b5c51fb6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.067753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d6641-e357-46ed-b2f4-c1b5c51fb6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.068025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d6641-e357-46ed-b2f4-c1b5c51fb6b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.068766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d6641-e357-46ed-b2f4-c1b5c51fb6b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:05.069063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d6641-e357-46ed-b2f4-c1b5c51fb6b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:05.070468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d6641-e357-46ed-b2f4-c1b5c51fb6b9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:05.087144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 342/668] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:05.087144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d6641-e357-46ed-b2f4-c1b5c51fb6b9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.100782 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:05.117212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dec0ae-50a3-4c9e-8924-d7b2238fc57d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.153656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c26e-4cc4-45cd-8557-71552466f54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:05.154500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c26e-4cc4-45cd-8557-71552466f54a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.154791 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c26e-4cc4-45cd-8557-71552466f54a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.155076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c26e-4cc4-45cd-8557-71552466f54a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.166076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33c26e-4cc4-45cd-8557-71552466f54a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:05.167610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33c26e-4cc4-45cd-8557-71552466f54a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2e33c26e-4cc4-45cd-8557-71552466f54a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:05.185794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:05.186978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:05.187875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:05.187875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:05.187875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:05.187875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:05.187875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 327/669] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:05.188170 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c26e-4cc4-45cd-8557-71552466f54a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.201488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d6641-e357-46ed-b2f4-c1b5c51fb6b9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.241980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16dfb2-b75c-4168-9db1-34a77dee7fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:05.243485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16dfb2-b75c-4168-9db1-34a77dee7fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.243952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16dfb2-b75c-4168-9db1-34a77dee7fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.244634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16dfb2-b75c-4168-9db1-34a77dee7fc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.257253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16dfb2-b75c-4168-9db1-34a77dee7fc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1d16dfb2-b75c-4168-9db1-34a77dee7fc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:05.267179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:05.268150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:05.268990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 343/670] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:05.269684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16dfb2-b75c-4168-9db1-34a77dee7fc0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.300853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c26e-4cc4-45cd-8557-71552466f54a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.309043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:05.340284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb27bfe-7807-4a15-953d-b35b83b3f5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:05.340872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb27bfe-7807-4a15-953d-b35b83b3f5cb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.341155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb27bfe-7807-4a15-953d-b35b83b3f5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.341437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb27bfe-7807-4a15-953d-b35b83b3f5cb None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.342266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb27bfe-7807-4a15-953d-b35b83b3f5cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:05.342582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb27bfe-7807-4a15-953d-b35b83b3f5cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:05.344046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb27bfe-7807-4a15-953d-b35b83b3f5cb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:05.347683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:05.357592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 328/671] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:05.358285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb27bfe-7807-4a15-953d-b35b83b3f5cb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.380588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16dfb2-b75c-4168-9db1-34a77dee7fc0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.414310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca991862-70f6-4e30-9e82-89376fddb815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:05.415124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca991862-70f6-4e30-9e82-89376fddb815 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.415428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca991862-70f6-4e30-9e82-89376fddb815 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.415707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca991862-70f6-4e30-9e82-89376fddb815 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.422449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca991862-70f6-4e30-9e82-89376fddb815 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:05.423919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca991862-70f6-4e30-9e82-89376fddb815 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-ca991862-70f6-4e30-9e82-89376fddb815 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:05.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:05.438975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:05.440108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:05.440108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:05.440108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:05.440108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:05.440108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:05.440108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:05.440108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:05.440108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:05.440108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 344/672] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:05.440108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca991862-70f6-4e30-9e82-89376fddb815 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.458814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb27bfe-7807-4a15-953d-b35b83b3f5cb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.492540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654841d5-f216-4949-a974-ad08f3cfd90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:05.493197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654841d5-f216-4949-a974-ad08f3cfd90e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.493483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654841d5-f216-4949-a974-ad08f3cfd90e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.493752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654841d5-f216-4949-a974-ad08f3cfd90e None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.494512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654841d5-f216-4949-a974-ad08f3cfd90e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:05.494820 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654841d5-f216-4949-a974-ad08f3cfd90e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:05.496304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654841d5-f216-4949-a974-ad08f3cfd90e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:05.507259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 329/673] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:05.508465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a9eb2-55eb-45bf-ba4e-b8ce04afa41b None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.508748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a9eb2-55eb-45bf-ba4e-b8ce04afa41b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.509026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a9eb2-55eb-45bf-ba4e-b8ce04afa41b None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.510522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 330/674] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:05.514914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a9eb2-55eb-45bf-ba4e-b8ce04afa41b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.537100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca991862-70f6-4e30-9e82-89376fddb815 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.542312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:05.566346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:05.573032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f03bb-9a29-4f90-ac70-578516161786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:05.573841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f03bb-9a29-4f90-ac70-578516161786 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.574172 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f03bb-9a29-4f90-ac70-578516161786 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.574434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f03bb-9a29-4f90-ac70-578516161786 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.575190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f03bb-9a29-4f90-ac70-578516161786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:05.575501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f03bb-9a29-4f90-ac70-578516161786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:05.576894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f03bb-9a29-4f90-ac70-578516161786 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:05.585482 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 345/675] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:05.586313 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f03bb-9a29-4f90-ac70-578516161786 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.617575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a9eb2-55eb-45bf-ba4e-b8ce04afa41b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.650674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c917a-a4eb-4bb2-ba3e-66f8f97f055e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:05.651342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c917a-a4eb-4bb2-ba3e-66f8f97f055e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.651631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c917a-a4eb-4bb2-ba3e-66f8f97f055e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.651972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c917a-a4eb-4bb2-ba3e-66f8f97f055e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.652809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c917a-a4eb-4bb2-ba3e-66f8f97f055e None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=d73fe529c7d342f895aabcb9a39aba78) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:05.653125 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c917a-a4eb-4bb2-ba3e-66f8f97f055e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:05.654624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c917a-a4eb-4bb2-ba3e-66f8f97f055e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:05.660477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 331/676] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/role_assignments?user.id=d73fe529c7d342f895aabcb9a39aba78&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:05.661255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c917a-a4eb-4bb2-ba3e-66f8f97f055e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.692598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f03bb-9a29-4f90-ac70-578516161786 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.741210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adad53-01f1-4597-bf34-bf989d29bb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:05.741907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adad53-01f1-4597-bf34-bf989d29bb2a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.742209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adad53-01f1-4597-bf34-bf989d29bb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.742489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adad53-01f1-4597-bf34-bf989d29bb2a None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.743288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adad53-01f1-4597-bf34-bf989d29bb2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:05.743636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adad53-01f1-4597-bf34-bf989d29bb2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:05.745087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adad53-01f1-4597-bf34-bf989d29bb2a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:05.759424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 346/677] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:05.760753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa18d5-0f96-4b10-8939-0c556ee77ea8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.761144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa18d5-0f96-4b10-8939-0c556ee77ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.761434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa18d5-0f96-4b10-8939-0c556ee77ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.781522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-92fa18d5-0f96-4b10-8939-0c556ee77ea8 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:05.782904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-92fa18d5-0f96-4b10-8939-0c556ee77ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:05.783516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:05.795254 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c917a-a4eb-4bb2-ba3e-66f8f97f055e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.833499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d6999-f1b9-47d9-bef9-1e7c60a49231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:05.834423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d6999-f1b9-47d9-bef9-1e7c60a49231 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:05.834770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d6999-f1b9-47d9-bef9-1e7c60a49231 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:05.835108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d6999-f1b9-47d9-bef9-1e7c60a49231 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:05.839322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:05.843008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d6999-f1b9-47d9-bef9-1e7c60a49231 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:05.845409 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d6999-f1b9-47d9-bef9-1e7c60a49231 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-036d6999-f1b9-47d9-bef9-1e7c60a49231 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:05.860579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:05.861529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:05.862584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:05.862584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:05.862584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:05.862584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:05.862584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:05.862584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:05.862584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:05.862584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:05.862584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 332/678] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:05.863058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d6999-f1b9-47d9-bef9-1e7c60a49231 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.864703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 347/679] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:52:05.865988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa18d5-0f96-4b10-8939-0c556ee77ea8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:05.983347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d6999-f1b9-47d9-bef9-1e7c60a49231 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:05.985373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa18d5-0f96-4b10-8939-0c556ee77ea8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:06.019703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad1259-0739-4632-9c24-ee40a0f6ea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:06.020514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad1259-0739-4632-9c24-ee40a0f6ea28 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:06.020766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04916-22fa-458b-b49e-2bae7456e7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:06.020879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad1259-0739-4632-9c24-ee40a0f6ea28 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:06.021060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad1259-0739-4632-9c24-ee40a0f6ea28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:06.021432 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04916-22fa-458b-b49e-2bae7456e7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:06.021718 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04916-22fa-458b-b49e-2bae7456e7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:06.021996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04916-22fa-458b-b49e-2bae7456e7c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:06.022873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef04916-22fa-458b-b49e-2bae7456e7c4 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=841e89a6caef4a089edc0f3b1df2fe9e) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:06.023370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef04916-22fa-458b-b49e-2bae7456e7c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:06.025158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef04916-22fa-458b-b49e-2bae7456e7c4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:06.032145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad1259-0739-4632-9c24-ee40a0f6ea28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:06.043138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 348/680] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/role_assignments?user.id=841e89a6caef4a089edc0f3b1df2fe9e&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:06.043138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04916-22fa-458b-b49e-2bae7456e7c4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:06.043427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad1259-0739-4632-9c24-ee40a0f6ea28 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-84ad1259-0739-4632-9c24-ee40a0f6ea28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:06.064591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:06.065615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:06.066544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:06.066544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:06.066544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:06.066544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:06.070648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 333/681] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:06.071726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad1259-0739-4632-9c24-ee40a0f6ea28 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:06.076083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:06.097514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:06.170805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04916-22fa-458b-b49e-2bae7456e7c4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:06.200675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad1259-0739-4632-9c24-ee40a0f6ea28 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:06.213261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e90b1-52a9-4596-b7d9-f4e0a9def4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:06.214338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e90b1-52a9-4596-b7d9-f4e0a9def4ae None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:06.214830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e90b1-52a9-4596-b7d9-f4e0a9def4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:06.215257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e90b1-52a9-4596-b7d9-f4e0a9def4ae None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:06.216071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e90b1-52a9-4596-b7d9-f4e0a9def4ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:06.216379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e90b1-52a9-4596-b7d9-f4e0a9def4ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:06.217845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e90b1-52a9-4596-b7d9-f4e0a9def4ae None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:06.232875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 349/682] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:06.233738 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e90b1-52a9-4596-b7d9-f4e0a9def4ae None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:06.248744 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fa791-030e-40b8-a29c-4a84fff52ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:06.249717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fa791-030e-40b8-a29c-4a84fff52ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:06.250109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fa791-030e-40b8-a29c-4a84fff52ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:06.250487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fa791-030e-40b8-a29c-4a84fff52ba3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:06.263906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64fa791-030e-40b8-a29c-4a84fff52ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f64fa791-030e-40b8-a29c-4a84fff52ba3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:06.272648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:06.273670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:06.274533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:06.275812 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 334/683] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:52:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:06.275812 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fa791-030e-40b8-a29c-4a84fff52ba3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:06.359527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e90b1-52a9-4596-b7d9-f4e0a9def4ae None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:06.360227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:06.369243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:06.404895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529a162-1e5d-4988-b6a1-4da03ae0ae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:06.405528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64fa791-030e-40b8-a29c-4a84fff52ba3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:06.405834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529a162-1e5d-4988-b6a1-4da03ae0ae93 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:06.406238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529a162-1e5d-4988-b6a1-4da03ae0ae93 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:06.406610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529a162-1e5d-4988-b6a1-4da03ae0ae93 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:06.407616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529a162-1e5d-4988-b6a1-4da03ae0ae93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:06.408829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529a162-1e5d-4988-b6a1-4da03ae0ae93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:06.410353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529a162-1e5d-4988-b6a1-4da03ae0ae93 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:06.428807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 350/684] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:06.434674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529a162-1e5d-4988-b6a1-4da03ae0ae93 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:06.453232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2faae-08b1-42b4-a921-ef630c78eac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:06.454317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2faae-08b1-42b4-a921-ef630c78eac3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:06.455173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2faae-08b1-42b4-a921-ef630c78eac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:06.455526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2faae-08b1-42b4-a921-ef630c78eac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:06.456735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2faae-08b1-42b4-a921-ef630c78eac3 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=e4bf4d9571d541bcbba87094198dc2ff) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:06.457725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2faae-08b1-42b4-a921-ef630c78eac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:06.459732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2faae-08b1-42b4-a921-ef630c78eac3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:06.470186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 335/685] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:06 2022] GET /identity/v3/role_assignments?user.id=e4bf4d9571d541bcbba87094198dc2ff&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:06.471059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2faae-08b1-42b4-a921-ef630c78eac3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:06.591233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2faae-08b1-42b4-a921-ef630c78eac3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:06.620054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:06.630287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bb486-748f-4856-9805-ca8bdf3b82b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:06.631618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb486-748f-4856-9805-ca8bdf3b82b7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:06.632228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb486-748f-4856-9805-ca8bdf3b82b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:06.632890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb486-748f-4856-9805-ca8bdf3b82b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:06.641950 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bb486-748f-4856-9805-ca8bdf3b82b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:06.650128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bb486-748f-4856-9805-ca8bdf3b82b7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-794bb486-748f-4856-9805-ca8bdf3b82b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:06.658412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:06.659340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:06.660492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 336/686] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jun 20 10:52:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:06.665328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bb486-748f-4856-9805-ca8bdf3b82b7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:06.669238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529a162-1e5d-4988-b6a1-4da03ae0ae93 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:06.708484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65bbbf-efb9-4f11-9d8c-bb530ef81470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:06.709476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65bbbf-efb9-4f11-9d8c-bb530ef81470 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:06.709904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65bbbf-efb9-4f11-9d8c-bb530ef81470 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:06.710340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65bbbf-efb9-4f11-9d8c-bb530ef81470 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:06.711326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65bbbf-efb9-4f11-9d8c-bb530ef81470 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:06.711784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65bbbf-efb9-4f11-9d8c-bb530ef81470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:06.713400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65bbbf-efb9-4f11-9d8c-bb530ef81470 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:06.723560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 351/687] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:52:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:06.730964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65bbbf-efb9-4f11-9d8c-bb530ef81470 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:06.772142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:06.787414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bb486-748f-4856-9805-ca8bdf3b82b7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:06.825364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46704acf-0e8f-4e9f-a385-29e701130ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:06.826050 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46704acf-0e8f-4e9f-a385-29e701130ece None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:06.826377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46704acf-0e8f-4e9f-a385-29e701130ece None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:06.826653 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46704acf-0e8f-4e9f-a385-29e701130ece None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:06.827510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46704acf-0e8f-4e9f-a385-29e701130ece None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:06.827825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46704acf-0e8f-4e9f-a385-29e701130ece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:06.829295 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46704acf-0e8f-4e9f-a385-29e701130ece None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:06.844656 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65bbbf-efb9-4f11-9d8c-bb530ef81470 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:06.849352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 337/688] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jun 20 10:52:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:06.854907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46704acf-0e8f-4e9f-a385-29e701130ece None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:06.882231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acbc25-3a26-42ef-89c0-74f3e8445ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:06.883322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acbc25-3a26-42ef-89c0-74f3e8445ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:06.883763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acbc25-3a26-42ef-89c0-74f3e8445ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:06.884179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acbc25-3a26-42ef-89c0-74f3e8445ed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:06.897515 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acbc25-3a26-42ef-89c0-74f3e8445ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:06.899433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acbc25-3a26-42ef-89c0-74f3e8445ed6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:06.908486 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-97acbc25-3a26-42ef-89c0-74f3e8445ed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:06.935200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:06.936157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:06.937028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:06.937028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:06.937028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:06.937028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:06.939484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 352/689] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:06.947574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acbc25-3a26-42ef-89c0-74f3e8445ed6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:06.978544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46704acf-0e8f-4e9f-a385-29e701130ece None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:07.025157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debb7be-3e5e-4374-a9e9-6cfb89b2962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:07.026227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debb7be-3e5e-4374-a9e9-6cfb89b2962c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:07.026818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debb7be-3e5e-4374-a9e9-6cfb89b2962c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:07.027446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debb7be-3e5e-4374-a9e9-6cfb89b2962c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:07.049916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:07.071627 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acbc25-3a26-42ef-89c0-74f3e8445ed6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:07.099696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debb7be-3e5e-4374-a9e9-6cfb89b2962c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:07.102920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debb7be-3e5e-4374-a9e9-6cfb89b2962c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:07.108634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e4347-6f28-4d1c-867b-0e5454ff3a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:07.109623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e4347-6f28-4d1c-867b-0e5454ff3a2f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:07.109910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e4347-6f28-4d1c-867b-0e5454ff3a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:07.110217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e4347-6f28-4d1c-867b-0e5454ff3a2f None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:07.110936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e4347-6f28-4d1c-867b-0e5454ff3a2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:07.111242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e4347-6f28-4d1c-867b-0e5454ff3a2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:07.112614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e4347-6f28-4d1c-867b-0e5454ff3a2f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:07.126557 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 353/690] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:07.139179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e4347-6f28-4d1c-867b-0e5454ff3a2f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:07.157833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:07.213834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 338/691] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 10:52:07.219741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debb7be-3e5e-4374-a9e9-6cfb89b2962c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:07.322173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debb7be-3e5e-4374-a9e9-6cfb89b2962c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:07.359841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c304c8-a8bf-4b2b-af88-5772ce7a4fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:07.361087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c304c8-a8bf-4b2b-af88-5772ce7a4fac None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:07.361569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c304c8-a8bf-4b2b-af88-5772ce7a4fac None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:07.362056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c304c8-a8bf-4b2b-af88-5772ce7a4fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:07.370340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e4347-6f28-4d1c-867b-0e5454ff3a2f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:07.374407 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c304c8-a8bf-4b2b-af88-5772ce7a4fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:07.376230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c304c8-a8bf-4b2b-af88-5772ce7a4fac None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:07.383927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-73c304c8-a8bf-4b2b-af88-5772ce7a4fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:07.395447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:07.396359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:07.397271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:07.397271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:07.397271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:07.397271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:07.399582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 339/692] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:07.405304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c304c8-a8bf-4b2b-af88-5772ce7a4fac None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:07.454843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffa7be-750f-4eb1-a1f9-2a3c515ffa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:07.455679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffa7be-750f-4eb1-a1f9-2a3c515ffa29 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:07.455965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffa7be-750f-4eb1-a1f9-2a3c515ffa29 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:07.456242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffa7be-750f-4eb1-a1f9-2a3c515ffa29 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:07.469326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffa7be-750f-4eb1-a1f9-2a3c515ffa29 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:07.471073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffa7be-750f-4eb1-a1f9-2a3c515ffa29 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:07.488652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a3ffa7be-750f-4eb1-a1f9-2a3c515ffa29 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:07.508149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:07.509149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:07.510073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 354/693] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:07.512963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c304c8-a8bf-4b2b-af88-5772ce7a4fac None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:07.519422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffa7be-750f-4eb1-a1f9-2a3c515ffa29 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:07.549162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ca5e8-116c-41d1-b1c5-a3eb0d9d365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:07.550287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ca5e8-116c-41d1-b1c5-a3eb0d9d365a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:07.550794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ca5e8-116c-41d1-b1c5-a3eb0d9d365a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:07.551262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ca5e8-116c-41d1-b1c5-a3eb0d9d365a None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:07.552179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ca5e8-116c-41d1-b1c5-a3eb0d9d365a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:07.552685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ca5e8-116c-41d1-b1c5-a3eb0d9d365a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:07.554301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ca5e8-116c-41d1-b1c5-a3eb0d9d365a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:07.566761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 340/694] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:07.572459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ca5e8-116c-41d1-b1c5-a3eb0d9d365a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:07.643462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:07.681134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ca5e8-116c-41d1-b1c5-a3eb0d9d365a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:07.717753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecc87a-671e-4653-a573-a513c102a7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:07.719037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecc87a-671e-4653-a573-a513c102a7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:07.719519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecc87a-671e-4653-a573-a513c102a7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:07.719988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecc87a-671e-4653-a573-a513c102a7e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:07.728333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecc87a-671e-4653-a573-a513c102a7e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:07.730149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecc87a-671e-4653-a573-a513c102a7e3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-26ecc87a-671e-4653-a573-a513c102a7e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:07.745941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:07.746925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:07.747803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:07.747803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:07.747803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:07.747803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:07.747803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:07.747803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:07.747803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:07.747803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:07.750457 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 341/695] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:07.756661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecc87a-671e-4653-a573-a513c102a7e3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:07.841122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffa7be-750f-4eb1-a1f9-2a3c515ffa29 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:07.865504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecc87a-671e-4653-a573-a513c102a7e3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:07.890516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:07.905697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbf3e6-d514-4bae-802c-c1cba684f882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:07.906781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbf3e6-d514-4bae-802c-c1cba684f882 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:07.908104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbf3e6-d514-4bae-802c-c1cba684f882 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:07.908582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbf3e6-d514-4bae-802c-c1cba684f882 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:07.909778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbf3e6-d514-4bae-802c-c1cba684f882 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:07.910827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbf3e6-d514-4bae-802c-c1cba684f882 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:07.912510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbf3e6-d514-4bae-802c-c1cba684f882 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:07.926634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 342/696] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:07.933623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbf3e6-d514-4bae-802c-c1cba684f882 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:07.950716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa138942-4ea4-4961-bcdf-de4a72267a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:07.951554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa138942-4ea4-4961-bcdf-de4a72267a03 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:07.951860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa138942-4ea4-4961-bcdf-de4a72267a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:07.952105 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa138942-4ea4-4961-bcdf-de4a72267a03 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:07.952940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa138942-4ea4-4961-bcdf-de4a72267a03 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:07.953251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa138942-4ea4-4961-bcdf-de4a72267a03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:07.954714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa138942-4ea4-4961-bcdf-de4a72267a03 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:07.977565 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 355/697] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jun 20 10:52:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:07.984338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309354a8-61c9-4682-9fa5-e35ed64ec53c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:07.984865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309354a8-61c9-4682-9fa5-e35ed64ec53c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:07.985341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309354a8-61c9-4682-9fa5-e35ed64ec53c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:07.987163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 356/698] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:07.988839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309354a8-61c9-4682-9fa5-e35ed64ec53c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:08.053272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:08.072969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbf3e6-d514-4bae-802c-c1cba684f882 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:08.101882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309354a8-61c9-4682-9fa5-e35ed64ec53c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:08.120116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e85a8-5a55-41d3-b601-3b3562288063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:08.121599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e85a8-5a55-41d3-b601-3b3562288063 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:08.122105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e85a8-5a55-41d3-b601-3b3562288063 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:08.122586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e85a8-5a55-41d3-b601-3b3562288063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:08.123619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e85a8-5a55-41d3-b601-3b3562288063 None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=37b1f13aa474493d83b3a16f76949fbc) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:08.124616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e85a8-5a55-41d3-b601-3b3562288063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:08.126925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e85a8-5a55-41d3-b601-3b3562288063 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:08.135758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 343/699] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:07 2022] GET /identity/v3/role_assignments?user.id=37b1f13aa474493d83b3a16f76949fbc&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:08.137313 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e892e-3a3d-4163-b961-ee9849fe7da6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:08.137842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e892e-3a3d-4163-b961-ee9849fe7da6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:08.138428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e892e-3a3d-4163-b961-ee9849fe7da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:08.151912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53fd40-39a5-4e7d-a645-312ca8597020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:08.153225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53fd40-39a5-4e7d-a645-312ca8597020 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:08.153509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53fd40-39a5-4e7d-a645-312ca8597020 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:08.153800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53fd40-39a5-4e7d-a645-312ca8597020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:08.158724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-060e892e-3a3d-4163-b961-ee9849fe7da6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:08.160399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-060e892e-3a3d-4163-b961-ee9849fe7da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:08.226515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:08.242816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 344/700] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:52:08.273982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e892e-3a3d-4163-b961-ee9849fe7da6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:08.339400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53fd40-39a5-4e7d-a645-312ca8597020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:08.341117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53fd40-39a5-4e7d-a645-312ca8597020 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:08.408281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e892e-3a3d-4163-b961-ee9849fe7da6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:08.419161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:08.454831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc1b24-460d-409b-a038-b8e37380cdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:08.456208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc1b24-460d-409b-a038-b8e37380cdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:08.456927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc1b24-460d-409b-a038-b8e37380cdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:08.457746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc1b24-460d-409b-a038-b8e37380cdb3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:08.459146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc1b24-460d-409b-a038-b8e37380cdb3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:08.461171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc1b24-460d-409b-a038-b8e37380cdb3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:08.469223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 357/701] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 10:52:08.469223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe41b9f-40f6-4fd2-99a7-dea0a212f17b None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:08.469223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe41b9f-40f6-4fd2-99a7-dea0a212f17b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:08.469223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe41b9f-40f6-4fd2-99a7-dea0a212f17b None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:08.472337 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 358/702] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:08.473105 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe41b9f-40f6-4fd2-99a7-dea0a212f17b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-c4cc1b24-460d-409b-a038-b8e37380cdb3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:08.482798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:08.483922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 10:52:08.484760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:08.487811 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 345/703] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:08 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:08.489481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93ca0f-10f9-495b-b435-d810fcee29b8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:08.489910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93ca0f-10f9-495b-b435-d810fcee29b8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:08.490542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93ca0f-10f9-495b-b435-d810fcee29b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:08.511880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-cf93ca0f-10f9-495b-b435-d810fcee29b8 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:08.514355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cf93ca0f-10f9-495b-b435-d810fcee29b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:08.550295 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:08.587139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe41b9f-40f6-4fd2-99a7-dea0a212f17b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:08.605723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 346/704] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:52:08.606840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93ca0f-10f9-495b-b435-d810fcee29b8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:08.647902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70e1aa-e0c7-42ee-b384-0e520d17fdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:08.648982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70e1aa-e0c7-42ee-b384-0e520d17fdef None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:08.649413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70e1aa-e0c7-42ee-b384-0e520d17fdef None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:08.649824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70e1aa-e0c7-42ee-b384-0e520d17fdef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:08.664563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70e1aa-e0c7-42ee-b384-0e520d17fdef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:08.666292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70e1aa-e0c7-42ee-b384-0e520d17fdef None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-4f70e1aa-e0c7-42ee-b384-0e520d17fdef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:08.685845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:08.686978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:08.688065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:08.688065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:08.688065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:08.688065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:08.689649 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 359/705] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:08.690644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70e1aa-e0c7-42ee-b384-0e520d17fdef None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:08.703466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:08.723588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93ca0f-10f9-495b-b435-d810fcee29b8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:08.770342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338d2eb-058a-4d04-a22e-b855a336e3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:08.771436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338d2eb-058a-4d04-a22e-b855a336e3bd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:08.771974 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338d2eb-058a-4d04-a22e-b855a336e3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:08.772451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338d2eb-058a-4d04-a22e-b855a336e3bd None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:08.773370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338d2eb-058a-4d04-a22e-b855a336e3bd None admin] RBAC: Adding query filter params (name=network) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:08.773898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338d2eb-058a-4d04-a22e-b855a336e3bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:08.776115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338d2eb-058a-4d04-a22e-b855a336e3bd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:08.784225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 347/706] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:08.785752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338d2eb-058a-4d04-a22e-b855a336e3bd None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:08.800356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70e1aa-e0c7-42ee-b384-0e520d17fdef None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:08.822241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:08.839302 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a507ba-5d5e-4b91-834c-c77d14305370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:08.840274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a507ba-5d5e-4b91-834c-c77d14305370 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:08.840524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a507ba-5d5e-4b91-834c-c77d14305370 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:08.840796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a507ba-5d5e-4b91-834c-c77d14305370 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:08.841459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a507ba-5d5e-4b91-834c-c77d14305370 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:08.842870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a507ba-5d5e-4b91-834c-c77d14305370 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a0a507ba-5d5e-4b91-834c-c77d14305370 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:08.857667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:08.858682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 10:52:08.859573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:08.863828 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 360/707] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Jun 20 10:52:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:08.864705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a507ba-5d5e-4b91-834c-c77d14305370 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:08.931119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338d2eb-058a-4d04-a22e-b855a336e3bd None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:08.947944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:08.975763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a507ba-5d5e-4b91-834c-c77d14305370 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:08.991043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2c89b-9f7c-4bcc-8d54-ce0fb3642b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:08.991723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2c89b-9f7c-4bcc-8d54-ce0fb3642b26 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:08.992007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2c89b-9f7c-4bcc-8d54-ce0fb3642b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:08.992284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2c89b-9f7c-4bcc-8d54-ce0fb3642b26 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:08.993021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2c89b-9f7c-4bcc-8d54-ce0fb3642b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:08.993330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2c89b-9f7c-4bcc-8d54-ce0fb3642b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:08.994760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2c89b-9f7c-4bcc-8d54-ce0fb3642b26 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:09.007806 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 348/708] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:09.008646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2c89b-9f7c-4bcc-8d54-ce0fb3642b26 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:09.018036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae0b95-ccd6-45a9-857b-939c4633a8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:09.019098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae0b95-ccd6-45a9-857b-939c4633a8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:09.019578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae0b95-ccd6-45a9-857b-939c4633a8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:09.020044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae0b95-ccd6-45a9-857b-939c4633a8c3 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:09.020957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae0b95-ccd6-45a9-857b-939c4633a8c3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:09.021490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae0b95-ccd6-45a9-857b-939c4633a8c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:09.023129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae0b95-ccd6-45a9-857b-939c4633a8c3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:09.029993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 361/709] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:09.031209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae0b95-ccd6-45a9-857b-939c4633a8c3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:09.110623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:09.125068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2c89b-9f7c-4bcc-8d54-ce0fb3642b26 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:09.137902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae0b95-ccd6-45a9-857b-939c4633a8c3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:09.163352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a762ec-9040-4422-8712-36322a4747aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:09.164449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a762ec-9040-4422-8712-36322a4747aa None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:09.164972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a762ec-9040-4422-8712-36322a4747aa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:09.165463 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a762ec-9040-4422-8712-36322a4747aa None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:09.166468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a762ec-9040-4422-8712-36322a4747aa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:09.167097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a762ec-9040-4422-8712-36322a4747aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:09.168887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a762ec-9040-4422-8712-36322a4747aa None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:09.173983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 349/710] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jun 20 10:52:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:09.183697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287078a-1b59-4b37-be68-42abca0cf538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:09.183697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287078a-1b59-4b37-be68-42abca0cf538 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:09.183697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287078a-1b59-4b37-be68-42abca0cf538 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:09.183697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287078a-1b59-4b37-be68-42abca0cf538 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:09.189750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a762ec-9040-4422-8712-36322a4747aa None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:09.189918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287078a-1b59-4b37-be68-42abca0cf538 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:09.191556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287078a-1b59-4b37-be68-42abca0cf538 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:09.193491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-f287078a-1b59-4b37-be68-42abca0cf538 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:09.208090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:09.209078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:09.209960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:09.209960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:09.209960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:09.209960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:09.209960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:09.209960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:09.209960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:09.209960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:09.209960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 362/711] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:09.215794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287078a-1b59-4b37-be68-42abca0cf538 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:09.331822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287078a-1b59-4b37-be68-42abca0cf538 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:09.360483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a762ec-9040-4422-8712-36322a4747aa None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:09.375531 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d3d4b-697e-42cd-86b7-36e422277d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:09.376303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d3d4b-697e-42cd-86b7-36e422277d12 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:09.376590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d3d4b-697e-42cd-86b7-36e422277d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:09.376871 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d3d4b-697e-42cd-86b7-36e422277d12 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:09.378072 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d3d4b-697e-42cd-86b7-36e422277d12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:09.379404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d3d4b-697e-42cd-86b7-36e422277d12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:09.381676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d3d4b-697e-42cd-86b7-36e422277d12 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:09.397974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 363/712] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:09.404223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d3d4b-697e-42cd-86b7-36e422277d12 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:09.406808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05d044-8334-45ee-85d9-8a75ac97a2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:09.407711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05d044-8334-45ee-85d9-8a75ac97a2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:09.408121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05d044-8334-45ee-85d9-8a75ac97a2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:09.408534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05d044-8334-45ee-85d9-8a75ac97a2d0 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:09.409350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05d044-8334-45ee-85d9-8a75ac97a2d0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:09.409751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05d044-8334-45ee-85d9-8a75ac97a2d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:09.411271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05d044-8334-45ee-85d9-8a75ac97a2d0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:09.422259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 350/713] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jun 20 10:52:09 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:09.442532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15f42a-2875-47ac-b5bd-326a2bf4a927 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:09.442825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15f42a-2875-47ac-b5bd-326a2bf4a927 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:09.443106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15f42a-2875-47ac-b5bd-326a2bf4a927 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:09.444878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 351/714] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:09.451164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f1a63-c4e5-4ef5-9e6c-054116256d0a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:09.451465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f1a63-c4e5-4ef5-9e6c-054116256d0a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:09.451716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f1a63-c4e5-4ef5-9e6c-054116256d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:09.472698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:09.491127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-817f1a63-c4e5-4ef5-9e6c-054116256d0a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:09.492338 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-817f1a63-c4e5-4ef5-9e6c-054116256d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:09.514531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:09.534841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d3d4b-697e-42cd-86b7-36e422277d12 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:09.574598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf58bc5-4f10-4121-994f-3dd927c68612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:09.575410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf58bc5-4f10-4121-994f-3dd927c68612 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:09.575689 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf58bc5-4f10-4121-994f-3dd927c68612 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:09.575966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf58bc5-4f10-4121-994f-3dd927c68612 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:09.576869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf58bc5-4f10-4121-994f-3dd927c68612 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=3c133005ca8d4478be8ff4acad5e88cc) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:09.577188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf58bc5-4f10-4121-994f-3dd927c68612 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:09.578810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf58bc5-4f10-4121-994f-3dd927c68612 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:09.587289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 364/715] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:09 2022] GET /identity/v3/role_assignments?user.id=3c133005ca8d4478be8ff4acad5e88cc&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:09.640702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 352/716] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:52:09.677670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf58bc5-4f10-4121-994f-3dd927c68612 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:09.789627 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf58bc5-4f10-4121-994f-3dd927c68612 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:09.815988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:09.833907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ba4a6-2f81-4299-a744-96872c33cce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:09.834751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ba4a6-2f81-4299-a744-96872c33cce1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:09.835027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ba4a6-2f81-4299-a744-96872c33cce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:09.835299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ba4a6-2f81-4299-a744-96872c33cce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:09.847002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ba4a6-2f81-4299-a744-96872c33cce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:09.848464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ba4a6-2f81-4299-a744-96872c33cce1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-cf3ba4a6-2f81-4299-a744-96872c33cce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:09.867837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:09.868794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:09.869742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:09.869742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:09.869742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:09.869742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:09.872073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 365/717] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:09.877976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f1a63-c4e5-4ef5-9e6c-054116256d0a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:09.985492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f1a63-c4e5-4ef5-9e6c-054116256d0a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:10.020390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6632a6-debb-418a-a63c-12b7d0a351ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:10.021068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6632a6-debb-418a-a63c-12b7d0a351ae None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:10.021352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6632a6-debb-418a-a63c-12b7d0a351ae None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:10.021629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6632a6-debb-418a-a63c-12b7d0a351ae None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:10.022395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6632a6-debb-418a-a63c-12b7d0a351ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:10.022703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6632a6-debb-418a-a63c-12b7d0a351ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:10.024098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6632a6-debb-418a-a63c-12b7d0a351ae None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:10.036276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 353/718] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:10.041710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ba4a6-2f81-4299-a744-96872c33cce1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:10.166442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ba4a6-2f81-4299-a744-96872c33cce1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:10.204417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4c48c-3522-47ba-add3-13d2b44e88b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:10.205411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c48c-3522-47ba-add3-13d2b44e88b8 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:10.205899 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c48c-3522-47ba-add3-13d2b44e88b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:10.206371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4c48c-3522-47ba-add3-13d2b44e88b8 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:10.207615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4c48c-3522-47ba-add3-13d2b44e88b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:10.209226 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4c48c-3522-47ba-add3-13d2b44e88b8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:10.235692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 366/719] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:52:10 2022] POST /identity/v3/users => generated 254 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 10:52:10.405595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb4557-2c7c-4837-a7a8-a3a2fbd7dbe2 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:10.406187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb4557-2c7c-4837-a7a8-a3a2fbd7dbe2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:10.406684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb4557-2c7c-4837-a7a8-a3a2fbd7dbe2 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:10.408586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 354/720] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:10.414262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb396c-2697-4778-b80e-4cf492d514e2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:10.414878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb396c-2697-4778-b80e-4cf492d514e2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:10.415376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb396c-2697-4778-b80e-4cf492d514e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:10.439231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5feb396c-2697-4778-b80e-4cf492d514e2 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:10.440847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5feb396c-2697-4778-b80e-4cf492d514e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:10.470943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:10.529846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 367/721] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:52:10.551764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb4557-2c7c-4837-a7a8-a3a2fbd7dbe2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:10.562143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:10.665529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb4557-2c7c-4837-a7a8-a3a2fbd7dbe2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:10.700817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fd312-3b50-4d60-ac42-450c353653cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:10.701623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fd312-3b50-4d60-ac42-450c353653cf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:10.701901 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fd312-3b50-4d60-ac42-450c353653cf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:10.702199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fd312-3b50-4d60-ac42-450c353653cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:10.712847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fd312-3b50-4d60-ac42-450c353653cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-348fd312-3b50-4d60-ac42-450c353653cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:10.720408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:10.721384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:10.722265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 355/722] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:10.727760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb396c-2697-4778-b80e-4cf492d514e2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:10.839510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb396c-2697-4778-b80e-4cf492d514e2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:10.874642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383a21e-f554-423a-ba20-e635239d8c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:10.875383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383a21e-f554-423a-ba20-e635239d8c4e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:10.875605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383a21e-f554-423a-ba20-e635239d8c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:10.875884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383a21e-f554-423a-ba20-e635239d8c4e None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:10.876641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383a21e-f554-423a-ba20-e635239d8c4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:10.876955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383a21e-f554-423a-ba20-e635239d8c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:10.878376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383a21e-f554-423a-ba20-e635239d8c4e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:10.887695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 368/723] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:10.892989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fd312-3b50-4d60-ac42-450c353653cf None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:10.993848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:11.035039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fd312-3b50-4d60-ac42-450c353653cf None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:11.073688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53592f8a-adc2-4e8a-a149-8efdbf339ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:11.074556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53592f8a-adc2-4e8a-a149-8efdbf339ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:11.074836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53592f8a-adc2-4e8a-a149-8efdbf339ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:11.075108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53592f8a-adc2-4e8a-a149-8efdbf339ae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:11.087564 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53592f8a-adc2-4e8a-a149-8efdbf339ae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:11.089118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53592f8a-adc2-4e8a-a149-8efdbf339ae5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-53592f8a-adc2-4e8a-a149-8efdbf339ae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:11.115645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:11.116687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:11.117578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:11.117578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:11.117578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:11.117578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:11.117578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 356/724] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:11.122946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383a21e-f554-423a-ba20-e635239d8c4e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:11.163031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:11.244173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383a21e-f554-423a-ba20-e635239d8c4e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:11.280809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc92f2-77bb-46da-a4ce-7955fe98f848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:11.281492 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc92f2-77bb-46da-a4ce-7955fe98f848 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:11.281779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc92f2-77bb-46da-a4ce-7955fe98f848 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:11.282050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc92f2-77bb-46da-a4ce-7955fe98f848 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:11.282815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc92f2-77bb-46da-a4ce-7955fe98f848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:11.283126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc92f2-77bb-46da-a4ce-7955fe98f848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:11.284516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc92f2-77bb-46da-a4ce-7955fe98f848 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:11.299195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 369/725] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:11.304939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53592f8a-adc2-4e8a-a149-8efdbf339ae5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:11.444708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53592f8a-adc2-4e8a-a149-8efdbf339ae5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:11.474773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:11.499799 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b8e93-1cb1-4d33-a06a-d8b35c053b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:11.500805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b8e93-1cb1-4d33-a06a-d8b35c053b1b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:11.501265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b8e93-1cb1-4d33-a06a-d8b35c053b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:11.501595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b8e93-1cb1-4d33-a06a-d8b35c053b1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:11.513613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b8e93-1cb1-4d33-a06a-d8b35c053b1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:11.515278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b8e93-1cb1-4d33-a06a-d8b35c053b1b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-fc7b8e93-1cb1-4d33-a06a-d8b35c053b1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:11.538704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:11.539711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:11.540634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 357/726] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jun 20 10:52:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:11.545990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc92f2-77bb-46da-a4ce-7955fe98f848 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:11.653288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:11.680477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc92f2-77bb-46da-a4ce-7955fe98f848 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:11.689058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d770e-8dfd-4dd9-9c59-e3f3b357b30e None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:11.689352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d770e-8dfd-4dd9-9c59-e3f3b357b30e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:11.689637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d770e-8dfd-4dd9-9c59-e3f3b357b30e None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:11.691300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 358/727] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:11.707831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c22b5-87aa-451c-90ae-2a746c86953d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:11.708109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c22b5-87aa-451c-90ae-2a746c86953d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:11.708388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c22b5-87aa-451c-90ae-2a746c86953d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:11.719073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756611b-f989-4c2c-87fc-1fec8398b2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:11.719762 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756611b-f989-4c2c-87fc-1fec8398b2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:11.720048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756611b-f989-4c2c-87fc-1fec8398b2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:11.720325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756611b-f989-4c2c-87fc-1fec8398b2d8 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:11.721171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756611b-f989-4c2c-87fc-1fec8398b2d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:11.721483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756611b-f989-4c2c-87fc-1fec8398b2d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:11.722954 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756611b-f989-4c2c-87fc-1fec8398b2d8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:11.742340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-df6c22b5-87aa-451c-90ae-2a746c86953d None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:11.745297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-df6c22b5-87aa-451c-90ae-2a746c86953d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:11.760415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 370/728] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jun 20 10:52:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:11.765469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756611b-f989-4c2c-87fc-1fec8398b2d8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:11.879867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 359/729] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 10:52:11.891666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756611b-f989-4c2c-87fc-1fec8398b2d8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:11.915245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c22b5-87aa-451c-90ae-2a746c86953d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:11.945079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14510217-b7d9-435c-8252-c292d65a7e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:11.946742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14510217-b7d9-435c-8252-c292d65a7e24 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:11.947239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14510217-b7d9-435c-8252-c292d65a7e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:11.947705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14510217-b7d9-435c-8252-c292d65a7e24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:11.958997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:11.961889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14510217-b7d9-435c-8252-c292d65a7e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:11.963595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14510217-b7d9-435c-8252-c292d65a7e24 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-14510217-b7d9-435c-8252-c292d65a7e24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:11.985449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:11.986438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:11.987307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:11.987307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:11.987307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:11.987307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:11.989715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 371/730] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:11.996547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14510217-b7d9-435c-8252-c292d65a7e24 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:12.078124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c22b5-87aa-451c-90ae-2a746c86953d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:12.089568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:12.110373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14510217-b7d9-435c-8252-c292d65a7e24 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:12.130153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0e206-a61a-4acf-9b7a-e4497d62471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:12.131137 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0e206-a61a-4acf-9b7a-e4497d62471f None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.131574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0e206-a61a-4acf-9b7a-e4497d62471f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.131990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0e206-a61a-4acf-9b7a-e4497d62471f None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.132868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0e206-a61a-4acf-9b7a-e4497d62471f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:12.134491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0e206-a61a-4acf-9b7a-e4497d62471f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:12.153624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 360/731] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:52:11 2022] POST /identity/v3/services => generated 240 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:12.155367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c6ddb-1449-4a64-8012-11ff2e4c866c None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.155826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c6ddb-1449-4a64-8012-11ff2e4c866c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.156290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c6ddb-1449-4a64-8012-11ff2e4c866c None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.157977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 361/732] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:12.178632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fac32-3635-4ce4-a59c-78a10d8bc768 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.179223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fac32-3635-4ce4-a59c-78a10d8bc768 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.180153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fac32-3635-4ce4-a59c-78a10d8bc768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.208948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47accfa3-5eaf-4870-8958-0fcf4b512985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:12.209853 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47accfa3-5eaf-4870-8958-0fcf4b512985 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.210197 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47accfa3-5eaf-4870-8958-0fcf4b512985 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.210485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47accfa3-5eaf-4870-8958-0fcf4b512985 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.211227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47accfa3-5eaf-4870-8958-0fcf4b512985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:12.211544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47accfa3-5eaf-4870-8958-0fcf4b512985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:12.213041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47accfa3-5eaf-4870-8958-0fcf4b512985 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:12.219294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-308fac32-3635-4ce4-a59c-78a10d8bc768 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:12.220503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-308fac32-3635-4ce4-a59c-78a10d8bc768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:12.227077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 372/733] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:12.233764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47accfa3-5eaf-4870-8958-0fcf4b512985 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:12.333495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:12.359620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 362/734] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:12 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 10:52:12.392635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fac32-3635-4ce4-a59c-78a10d8bc768 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:12.413906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47accfa3-5eaf-4870-8958-0fcf4b512985 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:12.463863 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab027099-d137-4a56-9dfe-e38ba96d57a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:12.466184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027099-d137-4a56-9dfe-e38ba96d57a9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.467273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027099-d137-4a56-9dfe-e38ba96d57a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.468470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027099-d137-4a56-9dfe-e38ba96d57a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.478860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab027099-d137-4a56-9dfe-e38ba96d57a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:12.481378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab027099-d137-4a56-9dfe-e38ba96d57a9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-ab027099-d137-4a56-9dfe-e38ba96d57a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:12.498902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:12.499870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:12.501272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:12.501272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:12.501272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:12.501272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:12.501272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:12.501272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:12.501272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:12.501272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:12.501272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 373/735] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:12.501751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b978b-3f5e-4930-8d66-b06085bef799 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.502043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b978b-3f5e-4930-8d66-b06085bef799 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.507468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b978b-3f5e-4930-8d66-b06085bef799 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.508048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 374/736] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:12.509253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b978b-3f5e-4930-8d66-b06085bef799 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:12.526666 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:12.529380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fac32-3635-4ce4-a59c-78a10d8bc768 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:12.568475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90bf01-fd75-4939-b864-5231c9973398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:12.569257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90bf01-fd75-4939-b864-5231c9973398 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.569616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90bf01-fd75-4939-b864-5231c9973398 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.569965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90bf01-fd75-4939-b864-5231c9973398 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.570823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90bf01-fd75-4939-b864-5231c9973398 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:12.572298 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90bf01-fd75-4939-b864-5231c9973398 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:12.590960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 363/737] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:52:12 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 10:52:12.592610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a7fe5-8616-4986-9465-3682aa6ac601 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.592973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a7fe5-8616-4986-9465-3682aa6ac601 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.593321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a7fe5-8616-4986-9465-3682aa6ac601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.619922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-595a7fe5-8616-4986-9465-3682aa6ac601 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:12.621565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-595a7fe5-8616-4986-9465-3682aa6ac601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:12.647136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b978b-3f5e-4930-8d66-b06085bef799 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:12.650885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:12.695041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778215b-f442-4a43-aafd-15b994b863ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:12.696135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778215b-f442-4a43-aafd-15b994b863ad None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.696596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778215b-f442-4a43-aafd-15b994b863ad None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.697043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778215b-f442-4a43-aafd-15b994b863ad None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.698074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778215b-f442-4a43-aafd-15b994b863ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:12.698679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778215b-f442-4a43-aafd-15b994b863ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:12.701293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778215b-f442-4a43-aafd-15b994b863ad None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:12.718119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 364/738] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:12.718315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 375/739] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:12.723591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a7fe5-8616-4986-9465-3682aa6ac601 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:12.738890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778215b-f442-4a43-aafd-15b994b863ad None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:12.856098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:12.871814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a7fe5-8616-4986-9465-3682aa6ac601 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:12.887807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778215b-f442-4a43-aafd-15b994b863ad None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:12.915576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976b8f-a8fb-45a6-9bcc-6ee6fe9fe335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:12.916510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976b8f-a8fb-45a6-9bcc-6ee6fe9fe335 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.916800 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976b8f-a8fb-45a6-9bcc-6ee6fe9fe335 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.917482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976b8f-a8fb-45a6-9bcc-6ee6fe9fe335 None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/37b1f13aa474493d83b3a16f76949fbc/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.919328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f976b8f-a8fb-45a6-9bcc-6ee6fe9fe335 None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=37b1f13aa474493d83b3a16f76949fbc, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:12.920916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f976b8f-a8fb-45a6-9bcc-6ee6fe9fe335 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:12.924146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4d78b-4354-468b-b61b-abaddfb2b8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:12.924953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4d78b-4354-468b-b61b-abaddfb2b8ba None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:12.925242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4d78b-4354-468b-b61b-abaddfb2b8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:12.925636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4d78b-4354-468b-b61b-abaddfb2b8ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:12.940879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4d78b-4354-468b-b61b-abaddfb2b8ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:12.942816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 365/740] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:52:12 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/37b1f13aa474493d83b3a16f76949fbc/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-3ba4d78b-4354-468b-b61b-abaddfb2b8ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:12.949975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:12.951039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:12.951869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 376/741] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:12.959253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976b8f-a8fb-45a6-9bcc-6ee6fe9fe335 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:13.083674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976b8f-a8fb-45a6-9bcc-6ee6fe9fe335 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:13.130754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04554889-7764-486c-805e-946e570a00d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:13.131803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04554889-7764-486c-805e-946e570a00d3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:13.132282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04554889-7764-486c-805e-946e570a00d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:13.132750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04554889-7764-486c-805e-946e570a00d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:13.133747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04554889-7764-486c-805e-946e570a00d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:13.134307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04554889-7764-486c-805e-946e570a00d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:13.135991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04554889-7764-486c-805e-946e570a00d3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:13.146122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 366/742] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:13.151785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4d78b-4354-468b-b61b-abaddfb2b8ba None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:13.179100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:13.256392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4d78b-4354-468b-b61b-abaddfb2b8ba None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:13.292280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41817e4a-a268-4f72-ac19-1578c7448a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:13.293350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41817e4a-a268-4f72-ac19-1578c7448a10 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:13.293731 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41817e4a-a268-4f72-ac19-1578c7448a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:13.294116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41817e4a-a268-4f72-ac19-1578c7448a10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:13.300228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a733a8-3bce-4a69-a784-58e67d1d9093 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:13.300646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a733a8-3bce-4a69-a784-58e67d1d9093 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:13.301019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a733a8-3bce-4a69-a784-58e67d1d9093 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:13.302790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 367/743] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:13.313240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41817e4a-a268-4f72-ac19-1578c7448a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:13.313817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1022-9886-4d58-8afe-41c1caa77051 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:13.314214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1022-9886-4d58-8afe-41c1caa77051 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:13.314413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1022-9886-4d58-8afe-41c1caa77051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:13.314739 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41817e4a-a268-4f72-ac19-1578c7448a10 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-41817e4a-a268-4f72-ac19-1578c7448a10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:13.334586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:13.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:13.336415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:13.336415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:13.336415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:13.336415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:13.336415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 377/744] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:13.342456 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41817e4a-a268-4f72-ac19-1578c7448a10 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:13.359194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-918f1022-9886-4d58-8afe-41c1caa77051 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:13.360459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-918f1022-9886-4d58-8afe-41c1caa77051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:13.372601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:13.424572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:13.450475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41817e4a-a268-4f72-ac19-1578c7448a10 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:13.492288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7719ad-567a-4a3d-82f8-91a4b2cb181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:13.492962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7719ad-567a-4a3d-82f8-91a4b2cb181f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:13.495028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7719ad-567a-4a3d-82f8-91a4b2cb181f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:13.495028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7719ad-567a-4a3d-82f8-91a4b2cb181f None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:13.495028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7719ad-567a-4a3d-82f8-91a4b2cb181f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:13.496805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7719ad-567a-4a3d-82f8-91a4b2cb181f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:13.498254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7719ad-567a-4a3d-82f8-91a4b2cb181f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:13.525123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 378/745] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:13.533826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7719ad-567a-4a3d-82f8-91a4b2cb181f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:13.546420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 368/746] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:13.586514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f1022-9886-4d58-8afe-41c1caa77051 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:13.696273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7719ad-567a-4a3d-82f8-91a4b2cb181f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:13.728192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f1022-9886-4d58-8afe-41c1caa77051 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:13.734006 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2870f-3135-4de9-a7b5-b306b0833ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:13.735099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2870f-3135-4de9-a7b5-b306b0833ead None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:13.735585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2870f-3135-4de9-a7b5-b306b0833ead None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:13.736077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2870f-3135-4de9-a7b5-b306b0833ead None admin] PATH_INFO: `/v3/users/swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:13.739182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:13.744291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2870f-3135-4de9-a7b5-b306b0833ead None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:13.745946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2870f-3135-4de9-a7b5-b306b0833ead None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-b0b2870f-3135-4de9-a7b5-b306b0833ead None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:13.761827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:13.765339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:13.766394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 379/747] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:13.768687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2870f-3135-4de9-a7b5-b306b0833ead None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:13.773116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4ec22-2c76-4841-8831-965fd07299f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:13.774168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4ec22-2c76-4841-8831-965fd07299f1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:13.774643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4ec22-2c76-4841-8831-965fd07299f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:13.775087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4ec22-2c76-4841-8831-965fd07299f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:13.787390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4ec22-2c76-4841-8831-965fd07299f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-fda4ec22-2c76-4841-8831-965fd07299f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:13.796319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:13.797438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:13.798378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:13.802115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 369/748] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:52:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:13.802419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:13.809447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4ec22-2c76-4841-8831-965fd07299f1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:13.973387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2870f-3135-4de9-a7b5-b306b0833ead None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:14.002940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4ec22-2c76-4841-8831-965fd07299f1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:14.045033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52110741-0afa-494c-9b03-610265984f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:14.045721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52110741-0afa-494c-9b03-610265984f76 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:14.046003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52110741-0afa-494c-9b03-610265984f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:14.046294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52110741-0afa-494c-9b03-610265984f76 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:14.047261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52110741-0afa-494c-9b03-610265984f76 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:14.047570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52110741-0afa-494c-9b03-610265984f76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:14.049252 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52110741-0afa-494c-9b03-610265984f76 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:14.070149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 380/749] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jun 20 10:52:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:14.079720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dacf2-0672-4180-a341-d7406f5287d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:14.080489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dacf2-0672-4180-a341-d7406f5287d5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:14.080628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52110741-0afa-494c-9b03-610265984f76 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:14.080920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dacf2-0672-4180-a341-d7406f5287d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:14.081269 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dacf2-0672-4180-a341-d7406f5287d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:14.082608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dacf2-0672-4180-a341-d7406f5287d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:14.082983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dacf2-0672-4180-a341-d7406f5287d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:14.084440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dacf2-0672-4180-a341-d7406f5287d5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:14.093895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 370/750] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:52:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:14.099281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dacf2-0672-4180-a341-d7406f5287d5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:14.198552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:14.212134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:14.252782 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52110741-0afa-494c-9b03-610265984f76 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:14.299489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dacf2-0672-4180-a341-d7406f5287d5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:14.307888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba750f-3586-4237-b0a0-64f2015bf9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:14.308555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba750f-3586-4237-b0a0-64f2015bf9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:14.308838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba750f-3586-4237-b0a0-64f2015bf9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:14.309431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba750f-3586-4237-b0a0-64f2015bf9d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:14.351055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe6fe8-9da0-4bc4-8241-b5e36eb39fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:14.351938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe6fe8-9da0-4bc4-8241-b5e36eb39fcb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:14.352222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe6fe8-9da0-4bc4-8241-b5e36eb39fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:14.352548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe6fe8-9da0-4bc4-8241-b5e36eb39fcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:14.364441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe6fe8-9da0-4bc4-8241-b5e36eb39fcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:14.366013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe6fe8-9da0-4bc4-8241-b5e36eb39fcb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-1dfe6fe8-9da0-4bc4-8241-b5e36eb39fcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:14.384607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:14.385858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:14.387275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:14.387275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:14.387275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:14.387275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:14.390419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 371/751] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:14.391747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe6fe8-9da0-4bc4-8241-b5e36eb39fcb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:14.472149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba750f-3586-4237-b0a0-64f2015bf9d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:14.473780 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba750f-3586-4237-b0a0-64f2015bf9d8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:14.500162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe6fe8-9da0-4bc4-8241-b5e36eb39fcb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:14.518820 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:14.540725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126ab4e-ec4e-4a36-a275-97325d0d3666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:14.541408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ab4e-ec4e-4a36-a275-97325d0d3666 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:14.541686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ab4e-ec4e-4a36-a275-97325d0d3666 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:14.542080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126ab4e-ec4e-4a36-a275-97325d0d3666 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:14.542788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126ab4e-ec4e-4a36-a275-97325d0d3666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:14.543102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126ab4e-ec4e-4a36-a275-97325d0d3666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:14.544530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126ab4e-ec4e-4a36-a275-97325d0d3666 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:14.558601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 372/752] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:14.560055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d57e9-89b1-4c57-8b3a-3d5db946d9e0 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:14.560335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d57e9-89b1-4c57-8b3a-3d5db946d9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:14.560609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d57e9-89b1-4c57-8b3a-3d5db946d9e0 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:14.565903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 373/753] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:14.566881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d57e9-89b1-4c57-8b3a-3d5db946d9e0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:14.571653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:14.635903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 381/754] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:14.638547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa91202-df0d-4025-9f8e-d94dc16dde61 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:14.639320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa91202-df0d-4025-9f8e-d94dc16dde61 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:14.639775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa91202-df0d-4025-9f8e-d94dc16dde61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:14.661996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-dfa91202-df0d-4025-9f8e-d94dc16dde61 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:14.664279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dfa91202-df0d-4025-9f8e-d94dc16dde61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:14.675520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d57e9-89b1-4c57-8b3a-3d5db946d9e0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:14.711982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd7867-969a-464b-ab37-ed46d51de098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:14.712887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7867-969a-464b-ab37-ed46d51de098 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:14.713416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7867-969a-464b-ab37-ed46d51de098 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:14.713990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7867-969a-464b-ab37-ed46d51de098 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:14.722266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd7867-969a-464b-ab37-ed46d51de098 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:14.723821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd7867-969a-464b-ab37-ed46d51de098 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-cebd7867-969a-464b-ab37-ed46d51de098 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:14.740585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:14.741521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:14.742765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 374/755] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jun 20 10:52:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:14.744374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd7867-969a-464b-ab37-ed46d51de098 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:14.754363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 382/756] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:14.755515 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa91202-df0d-4025-9f8e-d94dc16dde61 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:14.800958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:14.873373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:14.875601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd7867-969a-464b-ab37-ed46d51de098 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:14.894675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa91202-df0d-4025-9f8e-d94dc16dde61 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:14.926112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b7081-2bae-4f31-a057-119a935ca168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:14.926112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b7081-2bae-4f31-a057-119a935ca168 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:14.926112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b7081-2bae-4f31-a057-119a935ca168 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:14.926112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b7081-2bae-4f31-a057-119a935ca168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:14.930775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef306a1-686b-46fd-a988-368723fbd77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:14.931451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef306a1-686b-46fd-a988-368723fbd77e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:14.931727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef306a1-686b-46fd-a988-368723fbd77e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:14.931999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef306a1-686b-46fd-a988-368723fbd77e None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:14.932946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef306a1-686b-46fd-a988-368723fbd77e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:14.933480 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef306a1-686b-46fd-a988-368723fbd77e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:14.935045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef306a1-686b-46fd-a988-368723fbd77e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:14.936913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b7081-2bae-4f31-a057-119a935ca168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:14.939100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b7081-2bae-4f31-a057-119a935ca168 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:14.955783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 383/757] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jun 20 10:52:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:14.956643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef306a1-686b-46fd-a988-368723fbd77e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-602b7081-2bae-4f31-a057-119a935ca168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:14.957670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:14.958515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:14.959405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:14.959405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:14.959405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:14.959405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:14.962349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 375/758] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:14.964537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b7081-2bae-4f31-a057-119a935ca168 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:15.057182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:15.074728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b7081-2bae-4f31-a057-119a935ca168 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:15.087907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef306a1-686b-46fd-a988-368723fbd77e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:15.111664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f61721-a888-4d23-889e-ffc0027c0d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:15.112368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f61721-a888-4d23-889e-ffc0027c0d12 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.112651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f61721-a888-4d23-889e-ffc0027c0d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.112924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f61721-a888-4d23-889e-ffc0027c0d12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.138060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e2217-8aff-4854-9ca2-8ead5f33c160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:15.139839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e2217-8aff-4854-9ca2-8ead5f33c160 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.140426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e2217-8aff-4854-9ca2-8ead5f33c160 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.140954 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e2217-8aff-4854-9ca2-8ead5f33c160 None admin] PATH_INFO: `/v3/services/network` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.142025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e2217-8aff-4854-9ca2-8ead5f33c160 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:15.144273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e2217-8aff-4854-9ca2-8ead5f33c160 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-664e2217-8aff-4854-9ca2-8ead5f33c160 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:15.162757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:15.164003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 10:52:15.165049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:15.172616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 384/759] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:14 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:15.173625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e2217-8aff-4854-9ca2-8ead5f33c160 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:15.185882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:15.192607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f61721-a888-4d23-889e-ffc0027c0d12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:15.194353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f61721-a888-4d23-889e-ffc0027c0d12 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:15.282442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:15.293120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e2217-8aff-4854-9ca2-8ead5f33c160 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:15.301147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 376/760] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:15.301841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f61721-a888-4d23-889e-ffc0027c0d12 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:15.335365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefc434-921e-4b54-9e2e-7b740b2b72dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:15.335962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefc434-921e-4b54-9e2e-7b740b2b72dd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.336403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefc434-921e-4b54-9e2e-7b740b2b72dd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.336684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefc434-921e-4b54-9e2e-7b740b2b72dd None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.338238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefc434-921e-4b54-9e2e-7b740b2b72dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:15.338679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefc434-921e-4b54-9e2e-7b740b2b72dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:15.340158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefc434-921e-4b54-9e2e-7b740b2b72dd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:15.354018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 385/761] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:15.355431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9647280-fb63-4f17-9fee-d3ed8cdf0094 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.355723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9647280-fb63-4f17-9fee-d3ed8cdf0094 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.356003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9647280-fb63-4f17-9fee-d3ed8cdf0094 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.357406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 386/762] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:15.357945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9647280-fb63-4f17-9fee-d3ed8cdf0094 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:15.409422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f61721-a888-4d23-889e-ffc0027c0d12 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:15.429107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:15.446034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b6de8-7f7a-419a-962d-e8452b85845b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:15.446828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6de8-7f7a-419a-962d-e8452b85845b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.447184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6de8-7f7a-419a-962d-e8452b85845b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.447582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6de8-7f7a-419a-962d-e8452b85845b None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.448364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b6de8-7f7a-419a-962d-e8452b85845b None admin] RBAC: Adding query filter params (name=network) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:15.448678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b6de8-7f7a-419a-962d-e8452b85845b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:15.450078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b6de8-7f7a-419a-962d-e8452b85845b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:15.459524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 377/763] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:15.459524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b6de8-7f7a-419a-962d-e8452b85845b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:15.473953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9647280-fb63-4f17-9fee-d3ed8cdf0094 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:15.509026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:15.510514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de7557-8aa6-42a2-b4d9-9c07aba913d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:15.511472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de7557-8aa6-42a2-b4d9-9c07aba913d3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.511756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de7557-8aa6-42a2-b4d9-9c07aba913d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.512034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de7557-8aa6-42a2-b4d9-9c07aba913d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.522343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de7557-8aa6-42a2-b4d9-9c07aba913d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:15.523802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de7557-8aa6-42a2-b4d9-9c07aba913d3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-19de7557-8aa6-42a2-b4d9-9c07aba913d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:15.546695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:15.547795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:15.548687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:15.548687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:15.548687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:15.548687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:15.548687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 387/764] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:15.549102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbc02a-6006-4cfd-b1a0-e839b4e8af19 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.549368 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbc02a-6006-4cfd-b1a0-e839b4e8af19 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.549632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbc02a-6006-4cfd-b1a0-e839b4e8af19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.562512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b6de8-7f7a-419a-962d-e8452b85845b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:15.569775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b7fbc02a-6006-4cfd-b1a0-e839b4e8af19 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:15.571593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b7fbc02a-6006-4cfd-b1a0-e839b4e8af19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:15.598428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb029a-f54d-429e-ad8d-908e30c08421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:15.599196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb029a-f54d-429e-ad8d-908e30c08421 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.599450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb029a-f54d-429e-ad8d-908e30c08421 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.599728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb029a-f54d-429e-ad8d-908e30c08421 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.606688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb029a-f54d-429e-ad8d-908e30c08421 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:15.608154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb029a-f54d-429e-ad8d-908e30c08421 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-3abb029a-f54d-429e-ad8d-908e30c08421 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:15.623119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:15.624134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:15.625157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:15.625157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:15.625157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:15.625157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:15.625157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:15.625157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:15.625157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:15.625157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:15.625157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 378/765] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:15.625157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb029a-f54d-429e-ad8d-908e30c08421 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:15.646839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 388/766] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:15.648180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787178e-c671-426f-a80e-799a5e1b339f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.648470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787178e-c671-426f-a80e-799a5e1b339f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.648748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787178e-c671-426f-a80e-799a5e1b339f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.654154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 389/767] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:15.654154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787178e-c671-426f-a80e-799a5e1b339f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:15.701530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:15.732181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb029a-f54d-429e-ad8d-908e30c08421 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:15.742544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:15.752601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787178e-c671-426f-a80e-799a5e1b339f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:15.766702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213a66c-58d2-422e-b86a-417054b1fa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:15.767375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213a66c-58d2-422e-b86a-417054b1fa52 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.767657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213a66c-58d2-422e-b86a-417054b1fa52 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.767932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213a66c-58d2-422e-b86a-417054b1fa52 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.768645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213a66c-58d2-422e-b86a-417054b1fa52 None admin] RBAC: Adding query filter params (type=network) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:15.768952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213a66c-58d2-422e-b86a-417054b1fa52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:15.770376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213a66c-58d2-422e-b86a-417054b1fa52 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:15.775547 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 379/768] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:15.776352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213a66c-58d2-422e-b86a-417054b1fa52 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:15.791362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62600cbc-4629-490d-a549-6428d81539c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:15.792061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62600cbc-4629-490d-a549-6428d81539c8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.792344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62600cbc-4629-490d-a549-6428d81539c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.792717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62600cbc-4629-490d-a549-6428d81539c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.793437 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62600cbc-4629-490d-a549-6428d81539c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:15.793749 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62600cbc-4629-490d-a549-6428d81539c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:15.795182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62600cbc-4629-490d-a549-6428d81539c8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:15.806471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 390/769] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:15.807850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25038978-44b0-4c77-86cf-421ed6be59de None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.808260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25038978-44b0-4c77-86cf-421ed6be59de None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.808550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25038978-44b0-4c77-86cf-421ed6be59de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.824289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-25038978-44b0-4c77-86cf-421ed6be59de None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:15.825488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-25038978-44b0-4c77-86cf-421ed6be59de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:15.891311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213a66c-58d2-422e-b86a-417054b1fa52 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:15.894301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 391/770] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:15.895142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25038978-44b0-4c77-86cf-421ed6be59de None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:15.936867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99049a6-6c63-4554-82be-04cb68239c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:15.937540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99049a6-6c63-4554-82be-04cb68239c90 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:15.937814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99049a6-6c63-4554-82be-04cb68239c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:15.938124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99049a6-6c63-4554-82be-04cb68239c90 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:15.938909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99049a6-6c63-4554-82be-04cb68239c90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:15.939215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99049a6-6c63-4554-82be-04cb68239c90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:15.940096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:15.940644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99049a6-6c63-4554-82be-04cb68239c90 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:15.954757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 380/771] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:15.955578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99049a6-6c63-4554-82be-04cb68239c90 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:15.993542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:16.008507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25038978-44b0-4c77-86cf-421ed6be59de None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.046780 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bcdce-9751-4eea-92b2-c6069652dc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.047601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bcdce-9751-4eea-92b2-c6069652dc91 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.047885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bcdce-9751-4eea-92b2-c6069652dc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.048164 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bcdce-9751-4eea-92b2-c6069652dc91 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.048820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bcdce-9751-4eea-92b2-c6069652dc91 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:16.050223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bcdce-9751-4eea-92b2-c6069652dc91 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:16.060621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99049a6-6c63-4554-82be-04cb68239c90 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-021bcdce-9751-4eea-92b2-c6069652dc91 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:16.062398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:16.063293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:16.064098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 392/772] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Jun 20 10:52:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:16.064950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bcdce-9751-4eea-92b2-c6069652dc91 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.097777 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bacbf-5550-4d60-a66b-48e01f9ca487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.098468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bacbf-5550-4d60-a66b-48e01f9ca487 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.098745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bacbf-5550-4d60-a66b-48e01f9ca487 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.099012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656bacbf-5550-4d60-a66b-48e01f9ca487 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.099805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bacbf-5550-4d60-a66b-48e01f9ca487 None admin] RBAC: Adding query filter params (interface=public, service_id=1fb74333343e4099948fef5e33cc35a1, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:16.100118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bacbf-5550-4d60-a66b-48e01f9ca487 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:16.101508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656bacbf-5550-4d60-a66b-48e01f9ca487 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:16.109376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 381/773] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jun 20 10:52:15 2022] GET /identity/v3/endpoints?service_id=1fb74333343e4099948fef5e33cc35a1&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:16.110170 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bacbf-5550-4d60-a66b-48e01f9ca487 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.170153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bcdce-9751-4eea-92b2-c6069652dc91 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.171643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:16.205477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45f50d-697f-44f4-b4e4-8dab01efbbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.206357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45f50d-697f-44f4-b4e4-8dab01efbbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.206625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45f50d-697f-44f4-b4e4-8dab01efbbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.208454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45f50d-697f-44f4-b4e4-8dab01efbbc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.214326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45f50d-697f-44f4-b4e4-8dab01efbbc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:16.214540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:16.216126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45f50d-697f-44f4-b4e4-8dab01efbbc4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:16.218997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656bacbf-5550-4d60-a66b-48e01f9ca487 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2e45f50d-697f-44f4-b4e4-8dab01efbbc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:16.230892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:16.232004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:16.237847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:16.237847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:16.237847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:16.237847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:16.237847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:16.237847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:16.237847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:16.237847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:16.237847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 393/774] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:16.237847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45f50d-697f-44f4-b4e4-8dab01efbbc4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.254477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bf2c1-af07-4332-829f-ae0c15e0abac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.255311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bf2c1-af07-4332-829f-ae0c15e0abac None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.255598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bf2c1-af07-4332-829f-ae0c15e0abac None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.255876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3bf2c1-af07-4332-829f-ae0c15e0abac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.266794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3bf2c1-af07-4332-829f-ae0c15e0abac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6f3bf2c1-af07-4332-829f-ae0c15e0abac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:16.274474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:16.275552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:16.276464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 382/775] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:16.277321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bf2c1-af07-4332-829f-ae0c15e0abac None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.337490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45f50d-697f-44f4-b4e4-8dab01efbbc4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.371604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ec7d4-9ad4-40ac-803e-95d2c15f3156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.372397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ec7d4-9ad4-40ac-803e-95d2c15f3156 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.372793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ec7d4-9ad4-40ac-803e-95d2c15f3156 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.373075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ec7d4-9ad4-40ac-803e-95d2c15f3156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.373912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ec7d4-9ad4-40ac-803e-95d2c15f3156 None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=3c133005ca8d4478be8ff4acad5e88cc) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:16.374260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ec7d4-9ad4-40ac-803e-95d2c15f3156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:16.375726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ec7d4-9ad4-40ac-803e-95d2c15f3156 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:16.377157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3bf2c1-af07-4332-829f-ae0c15e0abac None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.381424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 394/776] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/role_assignments?user.id=3c133005ca8d4478be8ff4acad5e88cc&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:16.382019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ec7d4-9ad4-40ac-803e-95d2c15f3156 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.415959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e1c3b-94c7-43ae-b527-c665134d2454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.416821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e1c3b-94c7-43ae-b527-c665134d2454 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.417661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e1c3b-94c7-43ae-b527-c665134d2454 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.417959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e1c3b-94c7-43ae-b527-c665134d2454 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.418730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e1c3b-94c7-43ae-b527-c665134d2454 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:16.419041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e1c3b-94c7-43ae-b527-c665134d2454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:16.420525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e1c3b-94c7-43ae-b527-c665134d2454 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:16.426915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:16.433901 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 383/777] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:16.433901 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e1c3b-94c7-43ae-b527-c665134d2454 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.465338 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:16.517629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ec7d4-9ad4-40ac-803e-95d2c15f3156 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.556953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e1c3b-94c7-43ae-b527-c665134d2454 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.558812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822ddca-3669-4f4e-a72f-5a45987a5b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.559825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822ddca-3669-4f4e-a72f-5a45987a5b63 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.560166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822ddca-3669-4f4e-a72f-5a45987a5b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.560522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822ddca-3669-4f4e-a72f-5a45987a5b63 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.561217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822ddca-3669-4f4e-a72f-5a45987a5b63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:16.562639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822ddca-3669-4f4e-a72f-5a45987a5b63 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:16.578156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 395/778] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:16.578156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822ddca-3669-4f4e-a72f-5a45987a5b63 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.612202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa94cde-f5e9-4981-93f5-0f69bc454a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.613056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa94cde-f5e9-4981-93f5-0f69bc454a35 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.613342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa94cde-f5e9-4981-93f5-0f69bc454a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.614240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa94cde-f5e9-4981-93f5-0f69bc454a35 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.615130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa94cde-f5e9-4981-93f5-0f69bc454a35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:16.615499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa94cde-f5e9-4981-93f5-0f69bc454a35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:16.619343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa94cde-f5e9-4981-93f5-0f69bc454a35 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:16.638187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 384/779] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:16.639974 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa94cde-f5e9-4981-93f5-0f69bc454a35 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.701695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822ddca-3669-4f4e-a72f-5a45987a5b63 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.718478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:16.754918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da821fe3-a904-4d6f-a1c6-67369eb4b0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.755604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821fe3-a904-4d6f-a1c6-67369eb4b0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.755892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821fe3-a904-4d6f-a1c6-67369eb4b0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.756170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821fe3-a904-4d6f-a1c6-67369eb4b0e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.756925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da821fe3-a904-4d6f-a1c6-67369eb4b0e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:16.757240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da821fe3-a904-4d6f-a1c6-67369eb4b0e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:16.757558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:16.758656 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da821fe3-a904-4d6f-a1c6-67369eb4b0e8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:16.769001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 396/780] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:16.769001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da821fe3-a904-4d6f-a1c6-67369eb4b0e8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.782338 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa94cde-f5e9-4981-93f5-0f69bc454a35 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.826485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10beff5-fc88-4d8c-bf94-ed800783a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.827327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10beff5-fc88-4d8c-bf94-ed800783a53f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.827702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10beff5-fc88-4d8c-bf94-ed800783a53f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.828068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10beff5-fc88-4d8c-bf94-ed800783a53f None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.828882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10beff5-fc88-4d8c-bf94-ed800783a53f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:16.829304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10beff5-fc88-4d8c-bf94-ed800783a53f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:16.830831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10beff5-fc88-4d8c-bf94-ed800783a53f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:16.837172 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 385/781] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:16.838120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10beff5-fc88-4d8c-bf94-ed800783a53f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.891902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da821fe3-a904-4d6f-a1c6-67369eb4b0e8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:16.941978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bab31-3fac-41f0-8438-582af432a8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:16.942690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bab31-3fac-41f0-8438-582af432a8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:16.942975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bab31-3fac-41f0-8438-582af432a8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:16.943250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bab31-3fac-41f0-8438-582af432a8b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:16.944088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bab31-3fac-41f0-8438-582af432a8b5 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=aca3b45e80764c3b8254e24d09ff0d4c) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:16.944400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bab31-3fac-41f0-8438-582af432a8b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:16.946292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bab31-3fac-41f0-8438-582af432a8b5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:16.962221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 397/782] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/role_assignments?user.id=aca3b45e80764c3b8254e24d09ff0d4c&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:16.962221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bab31-3fac-41f0-8438-582af432a8b5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:16.994683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:17.028646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10beff5-fc88-4d8c-bf94-ed800783a53f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:17.087675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bab31-3fac-41f0-8438-582af432a8b5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:17.133706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bfe73-dfe8-4a93-b6ec-41cd60297dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:17.135841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47769f92-d46b-415f-a5f3-3c4c09f45d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:17.136541 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47769f92-d46b-415f-a5f3-3c4c09f45d5f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:17.136975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47769f92-d46b-415f-a5f3-3c4c09f45d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:17.137142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47769f92-d46b-415f-a5f3-3c4c09f45d5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:17.137918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47769f92-d46b-415f-a5f3-3c4c09f45d5f None admin] RBAC: Adding query filter params (interface=public, service_id=5b914ea011464a529e093dc783fd079a, region_id=RegionOne) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:17.138256 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47769f92-d46b-415f-a5f3-3c4c09f45d5f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:17.139682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47769f92-d46b-415f-a5f3-3c4c09f45d5f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:17.142174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bfe73-dfe8-4a93-b6ec-41cd60297dcc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:17.142458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bfe73-dfe8-4a93-b6ec-41cd60297dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:17.142732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bfe73-dfe8-4a93-b6ec-41cd60297dcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:17.159766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bfe73-dfe8-4a93-b6ec-41cd60297dcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:17.161270 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bfe73-dfe8-4a93-b6ec-41cd60297dcc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:17.164788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 398/783] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/endpoints?service_id=5b914ea011464a529e093dc783fd079a&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:17.172806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47769f92-d46b-415f-a5f3-3c4c09f45d5f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:17.178009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5e4bfe73-dfe8-4a93-b6ec-41cd60297dcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:17.191002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:17.191957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:17.193446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:17.193446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:17.193446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:17.193446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:17.193446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 386/784] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:17.203144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bfe73-dfe8-4a93-b6ec-41cd60297dcc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:17.269403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:17.290320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47769f92-d46b-415f-a5f3-3c4c09f45d5f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:17.355793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1739fbe-29e9-4d6a-814b-ba003088d10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:17.356909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1739fbe-29e9-4d6a-814b-ba003088d10c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:17.358681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1739fbe-29e9-4d6a-814b-ba003088d10c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:17.359160 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1739fbe-29e9-4d6a-814b-ba003088d10c None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:17.360152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1739fbe-29e9-4d6a-814b-ba003088d10c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:17.362530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1739fbe-29e9-4d6a-814b-ba003088d10c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:17.376025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 399/785] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jun 20 10:52:17 2022] GET /identity/v3/services => generated 772 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:17.399311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bfe73-dfe8-4a93-b6ec-41cd60297dcc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:17.451765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a61e-60de-47ea-ae2e-ad1933c30edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:17.452890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a61e-60de-47ea-ae2e-ad1933c30edd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:17.453404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a61e-60de-47ea-ae2e-ad1933c30edd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:17.453923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a61e-60de-47ea-ae2e-ad1933c30edd None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:17.454907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a61e-60de-47ea-ae2e-ad1933c30edd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:17.455427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a61e-60de-47ea-ae2e-ad1933c30edd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:17.457387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a61e-60de-47ea-ae2e-ad1933c30edd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:17.471325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 387/786] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:17.476561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1739fbe-29e9-4d6a-814b-ba003088d10c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:17.626921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1739fbe-29e9-4d6a-814b-ba003088d10c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:17.665385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f72c91-4a40-48c3-a1aa-349b850be5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:17.666614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72c91-4a40-48c3-a1aa-349b850be5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:17.667152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72c91-4a40-48c3-a1aa-349b850be5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:17.667667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72c91-4a40-48c3-a1aa-349b850be5d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:17.676692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f72c91-4a40-48c3-a1aa-349b850be5d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:17.678530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f72c91-4a40-48c3-a1aa-349b850be5d2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-67f72c91-4a40-48c3-a1aa-349b850be5d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:17.693552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:17.694571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:17.695520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:17.695520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:17.695520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:17.695520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:17.695520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:17.695520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:17.695520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:17.695520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:17.695520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:17.695520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:17.697823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 400/787] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jun 20 10:52:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:17.703908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a61e-60de-47ea-ae2e-ad1933c30edd None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:17.770958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:17.825747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a61e-60de-47ea-ae2e-ad1933c30edd None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:17.871873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1361c8b-7e99-465c-b56a-6f394de157ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:17.872934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1361c8b-7e99-465c-b56a-6f394de157ac None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:17.873417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1361c8b-7e99-465c-b56a-6f394de157ac None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:17.873909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1361c8b-7e99-465c-b56a-6f394de157ac None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:17.874986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1361c8b-7e99-465c-b56a-6f394de157ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:17.875513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1361c8b-7e99-465c-b56a-6f394de157ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:17.877220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1361c8b-7e99-465c-b56a-6f394de157ac None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:17.899086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 388/788] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jun 20 10:52:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:17.908311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f72c91-4a40-48c3-a1aa-349b850be5d2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:17.919520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:18.065845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f72c91-4a40-48c3-a1aa-349b850be5d2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:18.116138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1bb40-1f93-4b0a-ab29-e4a3c02a8282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:18.116799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1bb40-1f93-4b0a-ab29-e4a3c02a8282 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:18.117082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1bb40-1f93-4b0a-ab29-e4a3c02a8282 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:18.117358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1bb40-1f93-4b0a-ab29-e4a3c02a8282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:18.214827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1bb40-1f93-4b0a-ab29-e4a3c02a8282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:18.216213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1bb40-1f93-4b0a-ab29-e4a3c02a8282 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:18.223999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:18.343428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 401/789] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:18.349603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1361c8b-7e99-465c-b56a-6f394de157ac None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:18.501112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1361c8b-7e99-465c-b56a-6f394de157ac None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:18.517878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:18.546285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd539a-a0d4-4532-b0f5-21046d665a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:18.547088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd539a-a0d4-4532-b0f5-21046d665a2b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:18.547368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd539a-a0d4-4532-b0f5-21046d665a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:18.547646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd539a-a0d4-4532-b0f5-21046d665a2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:18.565818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd539a-a0d4-4532-b0f5-21046d665a2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:18.567334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd539a-a0d4-4532-b0f5-21046d665a2b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6bbd539a-a0d4-4532-b0f5-21046d665a2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:18.590595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:18.591525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:18.592833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:18.592833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:18.592833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:18.592833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:18.592833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 389/790] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:18.599781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1bb40-1f93-4b0a-ab29-e4a3c02a8282 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:18.750208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1bb40-1f93-4b0a-ab29-e4a3c02a8282 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:18.755129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:18.798977 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2afd18-2efe-4114-81c0-f56f067bdcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:18.799649 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2afd18-2efe-4114-81c0-f56f067bdcd1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:18.799927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2afd18-2efe-4114-81c0-f56f067bdcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:18.800197 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2afd18-2efe-4114-81c0-f56f067bdcd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:18.800912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2afd18-2efe-4114-81c0-f56f067bdcd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:18.801219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2afd18-2efe-4114-81c0-f56f067bdcd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:18.802616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2afd18-2efe-4114-81c0-f56f067bdcd1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:18.815066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 402/791] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:18.820117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea757b-4fee-4feb-865d-b79eb37f9a26 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:18.820323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2afd18-2efe-4114-81c0-f56f067bdcd1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:18.820449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea757b-4fee-4feb-865d-b79eb37f9a26 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:18.820710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea757b-4fee-4feb-865d-b79eb37f9a26 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:18.822336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 390/792] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:18.827200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabb28c-aeab-42e7-b101-71f5bd8d6959 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:18.827483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabb28c-aeab-42e7-b101-71f5bd8d6959 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:18.827761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabb28c-aeab-42e7-b101-71f5bd8d6959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:18.856515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-acabb28c-aeab-42e7-b101-71f5bd8d6959 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:18.857698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-acabb28c-aeab-42e7-b101-71f5bd8d6959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:18.976046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2afd18-2efe-4114-81c0-f56f067bdcd1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:19.008863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 391/793] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:19.019366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf96b56-f386-4d2e-82cc-b3cd3dc9675b None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.019655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf96b56-f386-4d2e-82cc-b3cd3dc9675b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.019937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf96b56-f386-4d2e-82cc-b3cd3dc9675b None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.021104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07297b-d3cf-4f32-8c60-99461cc30735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:19.021550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 392/794] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:19.022457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07297b-d3cf-4f32-8c60-99461cc30735 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.023046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07297b-d3cf-4f32-8c60-99461cc30735 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.023517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07297b-d3cf-4f32-8c60-99461cc30735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.026886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5527c-15de-43c2-bfc6-e7047585f4da None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.027138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5527c-15de-43c2-bfc6-e7047585f4da None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.027416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5527c-15de-43c2-bfc6-e7047585f4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.031851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07297b-d3cf-4f32-8c60-99461cc30735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:19.033534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07297b-d3cf-4f32-8c60-99461cc30735 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6c07297b-d3cf-4f32-8c60-99461cc30735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:19.049083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:19.050014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:19.050921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:19.050921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:19.050921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:19.050921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:19.050921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:19.050921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:19.050921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:19.050921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:19.053501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 403/795] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:19.054566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07297b-d3cf-4f32-8c60-99461cc30735 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:19.056757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-44a5527c-15de-43c2-bfc6-e7047585f4da None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:19.058267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-44a5527c-15de-43c2-bfc6-e7047585f4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:19.081252 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:19.137600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 393/796] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:19.146105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5527c-15de-43c2-bfc6-e7047585f4da None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:19.168408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:19.176156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07297b-d3cf-4f32-8c60-99461cc30735 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:19.214459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff16faa-57bd-46cc-93f2-6256d0efd6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:19.215609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff16faa-57bd-46cc-93f2-6256d0efd6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.216494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff16faa-57bd-46cc-93f2-6256d0efd6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.216915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff16faa-57bd-46cc-93f2-6256d0efd6b3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.217720 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff16faa-57bd-46cc-93f2-6256d0efd6b3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:19.219325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff16faa-57bd-46cc-93f2-6256d0efd6b3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-fff16faa-57bd-46cc-93f2-6256d0efd6b3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:19.232214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:19.233453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 10:52:19.234423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:19.241527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 404/797] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:19 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:19.242683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff16faa-57bd-46cc-93f2-6256d0efd6b3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:19.246481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5527c-15de-43c2-bfc6-e7047585f4da None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:19.283296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00094d-f667-4fe4-8be7-bb3d30a18cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:19.284021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00094d-f667-4fe4-8be7-bb3d30a18cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.284303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00094d-f667-4fe4-8be7-bb3d30a18cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.284582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00094d-f667-4fe4-8be7-bb3d30a18cb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.285422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00094d-f667-4fe4-8be7-bb3d30a18cb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:19.285733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00094d-f667-4fe4-8be7-bb3d30a18cb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:19.287406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00094d-f667-4fe4-8be7-bb3d30a18cb6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:19.305867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 394/798] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:19.306912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00094d-f667-4fe4-8be7-bb3d30a18cb6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:19.341680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:19.354793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff16faa-57bd-46cc-93f2-6256d0efd6b3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:19.392502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fe078-b269-4a1f-9d57-3f904c8d2215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:19.393703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fe078-b269-4a1f-9d57-3f904c8d2215 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.394161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fe078-b269-4a1f-9d57-3f904c8d2215 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.394692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fe078-b269-4a1f-9d57-3f904c8d2215 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.405286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:19.407234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fe078-b269-4a1f-9d57-3f904c8d2215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-090fe078-b269-4a1f-9d57-3f904c8d2215 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:19.415414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:19.416579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:19.417688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:19.418377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00094d-f667-4fe4-8be7-bb3d30a18cb6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:19.424784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 405/799] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:19.425857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fe078-b269-4a1f-9d57-3f904c8d2215 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:19.455875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d86b0-abed-41d0-be2a-f568b9d480ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:19.456549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d86b0-abed-41d0-be2a-f568b9d480ed None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.456875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d86b0-abed-41d0-be2a-f568b9d480ed None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.457527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d86b0-abed-41d0-be2a-f568b9d480ed None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.458268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d86b0-abed-41d0-be2a-f568b9d480ed None admin] RBAC: Adding query filter params (name=network) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:19.458577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d86b0-abed-41d0-be2a-f568b9d480ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:19.460271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d86b0-abed-41d0-be2a-f568b9d480ed None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:19.470120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 395/800] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:19.470120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d86b0-abed-41d0-be2a-f568b9d480ed None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:19.532320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fe078-b269-4a1f-9d57-3f904c8d2215 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:19.565698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35cf8b-a005-42e8-a704-f5951de44717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:19.566319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35cf8b-a005-42e8-a704-f5951de44717 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.566604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35cf8b-a005-42e8-a704-f5951de44717 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.566882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35cf8b-a005-42e8-a704-f5951de44717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.567711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35cf8b-a005-42e8-a704-f5951de44717 None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=37b1f13aa474493d83b3a16f76949fbc) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:19.568025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35cf8b-a005-42e8-a704-f5951de44717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:19.569465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35cf8b-a005-42e8-a704-f5951de44717 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:19.575687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 406/801] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:19 2022] GET /identity/v3/role_assignments?user.id=37b1f13aa474493d83b3a16f76949fbc&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:19.576847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d86b0-abed-41d0-be2a-f568b9d480ed None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:19.577291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c1037-060f-49d4-b18c-9c5acbd50d10 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.577291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c1037-060f-49d4-b18c-9c5acbd50d10 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.577450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c1037-060f-49d4-b18c-9c5acbd50d10 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.583564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 407/802] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:19.584508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c1037-060f-49d4-b18c-9c5acbd50d10 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:19.605583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:19.627808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1817a-8be1-4fa3-b752-4367505bbeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:19.628878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1817a-8be1-4fa3-b752-4367505bbeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.629406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1817a-8be1-4fa3-b752-4367505bbeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.629861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1817a-8be1-4fa3-b752-4367505bbeb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.630821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1817a-8be1-4fa3-b752-4367505bbeb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:19.631303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1817a-8be1-4fa3-b752-4367505bbeb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:19.633128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1817a-8be1-4fa3-b752-4367505bbeb8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:19.643976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 396/803] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:19.645754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4aed5-45d4-4569-89fb-482ee91e86a0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.646330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4aed5-45d4-4569-89fb-482ee91e86a0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.646780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4aed5-45d4-4569-89fb-482ee91e86a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.665796 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bfd4aed5-45d4-4569-89fb-482ee91e86a0 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:19.668296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bfd4aed5-45d4-4569-89fb-482ee91e86a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:19.679888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:19.712826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c1037-060f-49d4-b18c-9c5acbd50d10 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:19.752150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 397/804] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:19.753351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4aed5-45d4-4569-89fb-482ee91e86a0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:19.764720 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781a6cd-9e99-4183-a1c1-12f4da858dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:19.766060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781a6cd-9e99-4183-a1c1-12f4da858dfa None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.766543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781a6cd-9e99-4183-a1c1-12f4da858dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.766986 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781a6cd-9e99-4183-a1c1-12f4da858dfa None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.768549 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781a6cd-9e99-4183-a1c1-12f4da858dfa None admin] RBAC: Adding query filter params (type=network) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:19.769060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781a6cd-9e99-4183-a1c1-12f4da858dfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:19.770867 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781a6cd-9e99-4183-a1c1-12f4da858dfa None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:19.779233 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 408/805] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:19.780081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781a6cd-9e99-4183-a1c1-12f4da858dfa None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:19.879406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4aed5-45d4-4569-89fb-482ee91e86a0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:19.884588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:19.916330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781a6cd-9e99-4183-a1c1-12f4da858dfa None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:19.922051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139abbb-831d-46da-b1e3-730e64ffbbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:19.923083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139abbb-831d-46da-b1e3-730e64ffbbad None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.923547 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139abbb-831d-46da-b1e3-730e64ffbbad None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.923962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f139abbb-831d-46da-b1e3-730e64ffbbad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.935582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139abbb-831d-46da-b1e3-730e64ffbbad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:19.937245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f139abbb-831d-46da-b1e3-730e64ffbbad None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:19.940351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:19.957309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bd820-6bd9-4c05-b19d-f2ed4c7b4e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:19.958148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bd820-6bd9-4c05-b19d-f2ed4c7b4e1a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.958439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bd820-6bd9-4c05-b19d-f2ed4c7b4e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-f139abbb-831d-46da-b1e3-730e64ffbbad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:19.958649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:19.959639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:19.960723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:19.960723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:19.960723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:19.960723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:19.960996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bd820-6bd9-4c05-b19d-f2ed4c7b4e1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.962874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 398/806] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:19.963924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139abbb-831d-46da-b1e3-730e64ffbbad None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:19.972955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71bd820-6bd9-4c05-b19d-f2ed4c7b4e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-b71bd820-6bd9-4c05-b19d-f2ed4c7b4e1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:19.981114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:19.982075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:19.982944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 409/807] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:52:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:19.983629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6c6bf-3346-4c59-b281-1f063a7c9917 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:19.984194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6c6bf-3346-4c59-b281-1f063a7c9917 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:19.984194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6c6bf-3346-4c59-b281-1f063a7c9917 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:19.985554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 410/808] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:19.986070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6c6bf-3346-4c59-b281-1f063a7c9917 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:20.082506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f139abbb-831d-46da-b1e3-730e64ffbbad None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.090355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6c6bf-3346-4c59-b281-1f063a7c9917 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.117135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a2faf-f13b-4c3b-9ce8-dd76feb1cf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:20.117807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2faf-f13b-4c3b-9ce8-dd76feb1cf4a None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.118112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2faf-f13b-4c3b-9ce8-dd76feb1cf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.118631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2faf-f13b-4c3b-9ce8-dd76feb1cf4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:20.119164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a2faf-f13b-4c3b-9ce8-dd76feb1cf4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:20.120823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a2faf-f13b-4c3b-9ce8-dd76feb1cf4a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:20.127219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3014aec-4ddf-4bee-8626-6f34c10b60bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:20.127894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014aec-4ddf-4bee-8626-6f34c10b60bb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.128173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014aec-4ddf-4bee-8626-6f34c10b60bb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.128495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014aec-4ddf-4bee-8626-6f34c10b60bb None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:20.129219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3014aec-4ddf-4bee-8626-6f34c10b60bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:20.129581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3014aec-4ddf-4bee-8626-6f34c10b60bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:20.130941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3014aec-4ddf-4bee-8626-6f34c10b60bb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:20.134023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:20.144855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 399/809] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jun 20 10:52:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:20.149586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a2faf-f13b-4c3b-9ce8-dd76feb1cf4a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:20.150237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 411/810] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:20.152150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00d4bf-d0e4-429a-9a39-a68fe9b9b190 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.152636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00d4bf-d0e4-429a-9a39-a68fe9b9b190 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.152982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00d4bf-d0e4-429a-9a39-a68fe9b9b190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:20.174970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-db00d4bf-d0e4-429a-9a39-a68fe9b9b190 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:20.176275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-db00d4bf-d0e4-429a-9a39-a68fe9b9b190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:20.225590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:20.266555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 412/811] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:20.267565 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00d4bf-d0e4-429a-9a39-a68fe9b9b190 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:20.289554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a2faf-f13b-4c3b-9ce8-dd76feb1cf4a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.334605 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767451eb-ee3f-48e1-982d-7e19f2fbb2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:20.335303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767451eb-ee3f-48e1-982d-7e19f2fbb2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.335585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767451eb-ee3f-48e1-982d-7e19f2fbb2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.335859 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767451eb-ee3f-48e1-982d-7e19f2fbb2b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:20.336631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767451eb-ee3f-48e1-982d-7e19f2fbb2b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:20.336946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767451eb-ee3f-48e1-982d-7e19f2fbb2b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:20.338402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767451eb-ee3f-48e1-982d-7e19f2fbb2b6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:20.348949 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 400/812] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:52:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:20.354593 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767451eb-ee3f-48e1-982d-7e19f2fbb2b6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:20.400353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00d4bf-d0e4-429a-9a39-a68fe9b9b190 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.432632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:20.451991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ef6c-da4b-4525-840c-5ac96e9a6bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:20.453577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ef6c-da4b-4525-840c-5ac96e9a6bbe None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.454504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ef6c-da4b-4525-840c-5ac96e9a6bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.455055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ef6c-da4b-4525-840c-5ac96e9a6bbe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:20.469305 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ef6c-da4b-4525-840c-5ac96e9a6bbe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:20.471189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ef6c-da4b-4525-840c-5ac96e9a6bbe None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:20.478436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767451eb-ee3f-48e1-982d-7e19f2fbb2b6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1378ef6c-da4b-4525-840c-5ac96e9a6bbe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:20.486131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:20.487054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 413/813] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:20.488912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ef6c-da4b-4525-840c-5ac96e9a6bbe None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:20.493387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:20.514995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d16911-7025-45b3-8a56-a6d5ae5ab48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:20.516085 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d16911-7025-45b3-8a56-a6d5ae5ab48f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.516515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d16911-7025-45b3-8a56-a6d5ae5ab48f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.516966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d16911-7025-45b3-8a56-a6d5ae5ab48f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:20.517634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d16911-7025-45b3-8a56-a6d5ae5ab48f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:20.519165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d16911-7025-45b3-8a56-a6d5ae5ab48f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-97d16911-7025-45b3-8a56-a6d5ae5ab48f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:20.531300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:20.532227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:20.533051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 401/814] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Jun 20 10:52:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:20.533919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d16911-7025-45b3-8a56-a6d5ae5ab48f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:20.598460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ef6c-da4b-4525-840c-5ac96e9a6bbe None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.635169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1016231-de13-4e35-a553-ce8350619f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:20.636065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1016231-de13-4e35-a553-ce8350619f17 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.636349 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1016231-de13-4e35-a553-ce8350619f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.636625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1016231-de13-4e35-a553-ce8350619f17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:20.641460 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d16911-7025-45b3-8a56-a6d5ae5ab48f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.646985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1016231-de13-4e35-a553-ce8350619f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:20.648446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1016231-de13-4e35-a553-ce8350619f17 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-b1016231-de13-4e35-a553-ce8350619f17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:20.666453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:20.667542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:20.668414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:20.668414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:20.668414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:20.668414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:20.672399 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 414/815] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:20.673116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1016231-de13-4e35-a553-ce8350619f17 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:20.679009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce1cac-cd32-490b-9449-e171f6a4e4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:20.679693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1cac-cd32-490b-9449-e171f6a4e4ed None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.679977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1cac-cd32-490b-9449-e171f6a4e4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.680254 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1cac-cd32-490b-9449-e171f6a4e4ed None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:20.681103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1cac-cd32-490b-9449-e171f6a4e4ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:20.681480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1cac-cd32-490b-9449-e171f6a4e4ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:20.683093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1cac-cd32-490b-9449-e171f6a4e4ed None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:20.700668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 402/816] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jun 20 10:52:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:20.701491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce1cac-cd32-490b-9449-e171f6a4e4ed None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:20.717210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:20.740855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:20.788267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1016231-de13-4e35-a553-ce8350619f17 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.809891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce1cac-cd32-490b-9449-e171f6a4e4ed None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.825776 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4394e6-f0a5-40fc-b83f-dd2d66df8fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:20.826593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4394e6-f0a5-40fc-b83f-dd2d66df8fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.826880 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4394e6-f0a5-40fc-b83f-dd2d66df8fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.827155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4394e6-f0a5-40fc-b83f-dd2d66df8fa9 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:20.828064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4394e6-f0a5-40fc-b83f-dd2d66df8fa9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:20.828545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4394e6-f0a5-40fc-b83f-dd2d66df8fa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:20.829994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4394e6-f0a5-40fc-b83f-dd2d66df8fa9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:20.841328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 415/817] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jun 20 10:52:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:20.842462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4394e6-f0a5-40fc-b83f-dd2d66df8fa9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:20.846746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188aa172-454a-4143-950a-beac25bac7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:20.847500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188aa172-454a-4143-950a-beac25bac7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.847790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188aa172-454a-4143-950a-beac25bac7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.848069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188aa172-454a-4143-950a-beac25bac7c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:20.848863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188aa172-454a-4143-950a-beac25bac7c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:20.849181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188aa172-454a-4143-950a-beac25bac7c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:20.850609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188aa172-454a-4143-950a-beac25bac7c7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:20.862014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 403/818] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:20.862868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188aa172-454a-4143-950a-beac25bac7c7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:20.952059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:20.955985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4394e6-f0a5-40fc-b83f-dd2d66df8fa9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.972555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:20.980395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188aa172-454a-4143-950a-beac25bac7c7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:20.993579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6cef-7a2a-434e-a03b-8f0fc9bda341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:20.994425 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6cef-7a2a-434e-a03b-8f0fc9bda341 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:20.994704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6cef-7a2a-434e-a03b-8f0fc9bda341 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:20.994979 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6cef-7a2a-434e-a03b-8f0fc9bda341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.006623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff6cef-7a2a-434e-a03b-8f0fc9bda341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.008301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff6cef-7a2a-434e-a03b-8f0fc9bda341 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:21.015472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fb88d-9232-40bb-8322-7bbf3d87e32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:21.016147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fb88d-9232-40bb-8322-7bbf3d87e32d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.016434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fb88d-9232-40bb-8322-7bbf3d87e32d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.016738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fb88d-9232-40bb-8322-7bbf3d87e32d None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.017430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fb88d-9232-40bb-8322-7bbf3d87e32d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:21.017740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fb88d-9232-40bb-8322-7bbf3d87e32d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.019165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fb88d-9232-40bb-8322-7bbf3d87e32d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:21.023870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 404/819] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Jun 20 10:52:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:21.024649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fb88d-9232-40bb-8322-7bbf3d87e32d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e2ff6cef-7a2a-434e-a03b-8f0fc9bda341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:21.029223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:21.030075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:21.031108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:21.031108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:21.031108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:21.031108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:21.032635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 416/820] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:21.033612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6cef-7a2a-434e-a03b-8f0fc9bda341 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:21.131291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fb88d-9232-40bb-8322-7bbf3d87e32d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:21.140785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6cef-7a2a-434e-a03b-8f0fc9bda341 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:21.168209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c0050-717d-4a71-8098-4b2eb26d79c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:21.169035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c0050-717d-4a71-8098-4b2eb26d79c4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.169324 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c0050-717d-4a71-8098-4b2eb26d79c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.169608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c0050-717d-4a71-8098-4b2eb26d79c4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.177654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a26b5-aa88-4436-b3a0-f0f6156e883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:21.177803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c0050-717d-4a71-8098-4b2eb26d79c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.178620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a26b5-aa88-4436-b3a0-f0f6156e883f None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.179059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a26b5-aa88-4436-b3a0-f0f6156e883f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.179306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c0050-717d-4a71-8098-4b2eb26d79c4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:21.179612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a26b5-aa88-4436-b3a0-f0f6156e883f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.180476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a26b5-aa88-4436-b3a0-f0f6156e883f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.181995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a26b5-aa88-4436-b3a0-f0f6156e883f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-a47c0050-717d-4a71-8098-4b2eb26d79c4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:21.194491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:21.195749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 405/821] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jun 20 10:52:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:21.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c0050-717d-4a71-8098-4b2eb26d79c4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:21.218201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 417/822] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jun 20 10:52:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:21.218201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a26b5-aa88-4436-b3a0-f0f6156e883f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:21.226950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:21.232651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:21.357018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a26b5-aa88-4436-b3a0-f0f6156e883f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:21.363219 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c0050-717d-4a71-8098-4b2eb26d79c4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:21.404747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526645c-1d00-4bd7-abd7-b43de45c09a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:21.405734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526645c-1d00-4bd7-abd7-b43de45c09a6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.406191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526645c-1d00-4bd7-abd7-b43de45c09a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.406585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43386d99-ae1b-4e6f-9bf3-bae029ad1161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:21.406758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526645c-1d00-4bd7-abd7-b43de45c09a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.407273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43386d99-ae1b-4e6f-9bf3-bae029ad1161 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.407555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43386d99-ae1b-4e6f-9bf3-bae029ad1161 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.407827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43386d99-ae1b-4e6f-9bf3-bae029ad1161 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.408330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526645c-1d00-4bd7-abd7-b43de45c09a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:21.408676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43386d99-ae1b-4e6f-9bf3-bae029ad1161 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:21.408932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526645c-1d00-4bd7-abd7-b43de45c09a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.409092 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43386d99-ae1b-4e6f-9bf3-bae029ad1161 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.410477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526645c-1d00-4bd7-abd7-b43de45c09a6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:21.410579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43386d99-ae1b-4e6f-9bf3-bae029ad1161 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:21.425684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 406/823] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:21.431044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 418/824] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jun 20 10:52:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:21.431603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526645c-1d00-4bd7-abd7-b43de45c09a6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:21.438224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43386d99-ae1b-4e6f-9bf3-bae029ad1161 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:21.503070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:21.533550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:21.545193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43386d99-ae1b-4e6f-9bf3-bae029ad1161 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:21.555383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526645c-1d00-4bd7-abd7-b43de45c09a6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:21.582635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67045db4-c0ac-4155-a883-0a08eb685c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:21.583736 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67045db4-c0ac-4155-a883-0a08eb685c55 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.584165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67045db4-c0ac-4155-a883-0a08eb685c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.584579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67045db4-c0ac-4155-a883-0a08eb685c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.595710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67045db4-c0ac-4155-a883-0a08eb685c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.596062 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196026ca-2ef2-4e02-820b-75d44140e3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:21.596884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196026ca-2ef2-4e02-820b-75d44140e3ff None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.597166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196026ca-2ef2-4e02-820b-75d44140e3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.597441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196026ca-2ef2-4e02-820b-75d44140e3ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.598145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67045db4-c0ac-4155-a883-0a08eb685c55 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:21.606041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196026ca-2ef2-4e02-820b-75d44140e3ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.608017 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196026ca-2ef2-4e02-820b-75d44140e3ff None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-67045db4-c0ac-4155-a883-0a08eb685c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:21.618509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:21.619467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:21.620397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:21.620397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:21.620397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:21.620397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:21.623225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 419/825] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-196026ca-2ef2-4e02-820b-75d44140e3ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:21.623396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:21.624211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:21.625075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:21.625075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:21.625075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:21.625075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:21.625075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:21.625075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:21.625075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:21.625075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:21.629385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 407/826] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:21.630428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67045db4-c0ac-4155-a883-0a08eb685c55 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:21.634800 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196026ca-2ef2-4e02-820b-75d44140e3ff None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:21.737053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67045db4-c0ac-4155-a883-0a08eb685c55 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:21.743452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196026ca-2ef2-4e02-820b-75d44140e3ff None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:21.752058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:21.777294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d2109-c44b-4f9a-b694-2cbd1c6ceeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:21.778333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d2109-c44b-4f9a-b694-2cbd1c6ceeab None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.778997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d2109-c44b-4f9a-b694-2cbd1c6ceeab None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.779830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d2109-c44b-4f9a-b694-2cbd1c6ceeab None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.781014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d2109-c44b-4f9a-b694-2cbd1c6ceeab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:21.781510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d2109-c44b-4f9a-b694-2cbd1c6ceeab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.783062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d2109-c44b-4f9a-b694-2cbd1c6ceeab None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:21.786298 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9d3ac-60fd-4f77-aa15-f12eb985a65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:21.786970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d3ac-60fd-4f77-aa15-f12eb985a65a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.787252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d3ac-60fd-4f77-aa15-f12eb985a65a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.787526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d3ac-60fd-4f77-aa15-f12eb985a65a None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.788317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9d3ac-60fd-4f77-aa15-f12eb985a65a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:21.788627 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9d3ac-60fd-4f77-aa15-f12eb985a65a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.790052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9d3ac-60fd-4f77-aa15-f12eb985a65a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:21.798695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 420/827] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:21.799070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988cb49-0524-44d3-b4d7-0faf66915230 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.799715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988cb49-0524-44d3-b4d7-0faf66915230 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.799715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988cb49-0524-44d3-b4d7-0faf66915230 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.804934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 421/828] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:21.805072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:21.805168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988cb49-0524-44d3-b4d7-0faf66915230 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:21.814133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 408/829] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:21.814133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99377bf-ea7e-4184-9a2b-5a78244b1bb4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.814133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99377bf-ea7e-4184-9a2b-5a78244b1bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.814133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99377bf-ea7e-4184-9a2b-5a78244b1bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.828277 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a99377bf-ea7e-4184-9a2b-5a78244b1bb4 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:21.829462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a99377bf-ea7e-4184-9a2b-5a78244b1bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:21.904709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 409/830] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:21.905882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99377bf-ea7e-4184-9a2b-5a78244b1bb4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:21.931226 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988cb49-0524-44d3-b4d7-0faf66915230 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:21.971421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb2d9a-2a63-4e5a-9ea6-362a2ce0e88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:21.972290 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb2d9a-2a63-4e5a-9ea6-362a2ce0e88e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:21.972706 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb2d9a-2a63-4e5a-9ea6-362a2ce0e88e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:21.972840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb2d9a-2a63-4e5a-9ea6-362a2ce0e88e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:21.980021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb2d9a-2a63-4e5a-9ea6-362a2ce0e88e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:21.981559 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb2d9a-2a63-4e5a-9ea6-362a2ce0e88e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:22.012517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99377bf-ea7e-4184-9a2b-5a78244b1bb4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:22.048762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:22.054375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f837cf-b38b-42ce-9a21-137d409633a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:22.055308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f837cf-b38b-42ce-9a21-137d409633a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:22.055596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f837cf-b38b-42ce-9a21-137d409633a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:22.055871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f837cf-b38b-42ce-9a21-137d409633a6 None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/3c133005ca8d4478be8ff4acad5e88cc/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:22.058411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f837cf-b38b-42ce-9a21-137d409633a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=3c133005ca8d4478be8ff4acad5e88cc, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:22.059894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f837cf-b38b-42ce-9a21-137d409633a6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:22.082301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 410/831] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:52:21 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/3c133005ca8d4478be8ff4acad5e88cc/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:52:22.082301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f837cf-b38b-42ce-9a21-137d409633a6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-68cb2d9a-2a63-4e5a-9ea6-362a2ce0e88e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:22.122612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:22.123636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:22.125148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:22.125148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:22.125148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:22.125148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:22.125148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:22.125148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:22.125148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:22.125148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:22.125148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 422/832] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:22.129522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb2d9a-2a63-4e5a-9ea6-362a2ce0e88e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:22.163625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:22.205803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f837cf-b38b-42ce-9a21-137d409633a6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:22.243914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed32376-2c90-404a-a3dc-e4921e420475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:22.244820 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed32376-2c90-404a-a3dc-e4921e420475 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:22.245167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed32376-2c90-404a-a3dc-e4921e420475 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:22.245572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed32376-2c90-404a-a3dc-e4921e420475 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:22.246281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed32376-2c90-404a-a3dc-e4921e420475 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:22.247853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed32376-2c90-404a-a3dc-e4921e420475 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:22.261660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb2d9a-2a63-4e5a-9ea6-362a2ce0e88e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-7ed32376-2c90-404a-a3dc-e4921e420475 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:22.261877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:22.262985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 10:52:22.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:22.267991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 411/833] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Jun 20 10:52:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:22.270198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed32376-2c90-404a-a3dc-e4921e420475 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:22.304953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0329f8-6b0d-4da2-8d42-fc2a30173125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:22.305532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0329f8-6b0d-4da2-8d42-fc2a30173125 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:22.305811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0329f8-6b0d-4da2-8d42-fc2a30173125 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:22.306102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0329f8-6b0d-4da2-8d42-fc2a30173125 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:22.307571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0329f8-6b0d-4da2-8d42-fc2a30173125 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:22.307571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0329f8-6b0d-4da2-8d42-fc2a30173125 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:22.309239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0329f8-6b0d-4da2-8d42-fc2a30173125 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:22.323741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 423/834] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:22.330016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0329f8-6b0d-4da2-8d42-fc2a30173125 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:22.338559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:22.388107 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed32376-2c90-404a-a3dc-e4921e420475 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:22.424627 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046c26-d0f6-4005-82e4-6eff62c9c1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:22.425401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046c26-d0f6-4005-82e4-6eff62c9c1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:22.425685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046c26-d0f6-4005-82e4-6eff62c9c1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:22.425963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046c26-d0f6-4005-82e4-6eff62c9c1c3 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:22.426704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5046c26-d0f6-4005-82e4-6eff62c9c1c3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:22.427014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5046c26-d0f6-4005-82e4-6eff62c9c1c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:22.428493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5046c26-d0f6-4005-82e4-6eff62c9c1c3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:22.431397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:22.434501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 412/835] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:22.439886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046c26-d0f6-4005-82e4-6eff62c9c1c3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:22.441833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0329f8-6b0d-4da2-8d42-fc2a30173125 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:22.481584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421f46b-da53-4eea-8f81-dec5159114cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:22.482560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421f46b-da53-4eea-8f81-dec5159114cf None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:22.482840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421f46b-da53-4eea-8f81-dec5159114cf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:22.483115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421f46b-da53-4eea-8f81-dec5159114cf None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/aca3b45e80764c3b8254e24d09ff0d4c/roles/2cf42fcd695a4696b148241bfd911357` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:22.499699 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421f46b-da53-4eea-8f81-dec5159114cf None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=aca3b45e80764c3b8254e24d09ff0d4c, role_id=2cf42fcd695a4696b148241bfd911357)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:22.501144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421f46b-da53-4eea-8f81-dec5159114cf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:22.520014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 424/836] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:52:22 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/aca3b45e80764c3b8254e24d09ff0d4c/roles/2cf42fcd695a4696b148241bfd911357 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 10:52:22.562181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046c26-d0f6-4005-82e4-6eff62c9c1c3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:22.583450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:22.603158 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bdfce-36a3-43e1-87c8-344eec146fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:22.603903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bdfce-36a3-43e1-87c8-344eec146fce None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:22.604190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bdfce-36a3-43e1-87c8-344eec146fce None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:22.604537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bdfce-36a3-43e1-87c8-344eec146fce None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:22.605280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604bdfce-36a3-43e1-87c8-344eec146fce None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:22.605671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604bdfce-36a3-43e1-87c8-344eec146fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:22.607503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604bdfce-36a3-43e1-87c8-344eec146fce None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:22.613488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 413/837] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:23.383062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1494b-f431-4d99-92fa-01eff2b76e39 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:23.383476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1494b-f431-4d99-92fa-01eff2b76e39 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:23.383644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1494b-f431-4d99-92fa-01eff2b76e39 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:23.385266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 425/838] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:23.390586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6e3d8-5014-46a6-a0f6-d27455863f5c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:23.391098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6e3d8-5014-46a6-a0f6-d27455863f5c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:23.391250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6e3d8-5014-46a6-a0f6-d27455863f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:23.408172 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9eb6e3d8-5014-46a6-a0f6-d27455863f5c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:23.409360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9eb6e3d8-5014-46a6-a0f6-d27455863f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:23.476764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 414/839] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:23.499223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1494b-f431-4d99-92fa-01eff2b76e39 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:23.599971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1494b-f431-4d99-92fa-01eff2b76e39 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:23.635068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de373128-d095-4981-9766-ab0abf56a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:23.636036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de373128-d095-4981-9766-ab0abf56a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:23.636693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de373128-d095-4981-9766-ab0abf56a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:23.636971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de373128-d095-4981-9766-ab0abf56a1d1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:23.637626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de373128-d095-4981-9766-ab0abf56a1d1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:23.639032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de373128-d095-4981-9766-ab0abf56a1d1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-de373128-d095-4981-9766-ab0abf56a1d1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:23.650793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:23.653034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:23.654951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 426/840] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:52:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:23.662109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6e3d8-5014-46a6-a0f6-d27455863f5c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:23.773737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6e3d8-5014-46a6-a0f6-d27455863f5c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:23.788134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:23.811126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df501c0-a91f-4c92-9238-52f5083a99c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:23.812160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df501c0-a91f-4c92-9238-52f5083a99c6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:23.812649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df501c0-a91f-4c92-9238-52f5083a99c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:23.813125 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df501c0-a91f-4c92-9238-52f5083a99c6 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:23.814127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df501c0-a91f-4c92-9238-52f5083a99c6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:23.814628 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df501c0-a91f-4c92-9238-52f5083a99c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:23.816209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df501c0-a91f-4c92-9238-52f5083a99c6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:23.821303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 415/841] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:52:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:23.826806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de373128-d095-4981-9766-ab0abf56a1d1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:23.836545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:23.932982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de373128-d095-4981-9766-ab0abf56a1d1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:23.966493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9789edb-8c90-40ce-ae3e-2935ec1b4524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:23.967404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9789edb-8c90-40ce-ae3e-2935ec1b4524 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:23.967829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9789edb-8c90-40ce-ae3e-2935ec1b4524 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:23.968246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9789edb-8c90-40ce-ae3e-2935ec1b4524 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:23.969090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9789edb-8c90-40ce-ae3e-2935ec1b4524 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:23.969562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9789edb-8c90-40ce-ae3e-2935ec1b4524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:23.971098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9789edb-8c90-40ce-ae3e-2935ec1b4524 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:23.976101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 427/842] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:52:23 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:24.277222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf8b2a-6e37-455d-9acd-e38516f7ba42 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:24.277517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf8b2a-6e37-455d-9acd-e38516f7ba42 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:24.277833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf8b2a-6e37-455d-9acd-e38516f7ba42 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:24.279571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 416/843] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:24.285025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b7c5d-bdde-420b-a7b2-24e2bdc8a168 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:24.285360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b7c5d-bdde-420b-a7b2-24e2bdc8a168 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:24.285637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b7c5d-bdde-420b-a7b2-24e2bdc8a168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:24.303501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-499b7c5d-bdde-420b-a7b2-24e2bdc8a168 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:24.304690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-499b7c5d-bdde-420b-a7b2-24e2bdc8a168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:24.371197 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:24.380979 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 428/844] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:24.404735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf8b2a-6e37-455d-9acd-e38516f7ba42 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:24.515281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf8b2a-6e37-455d-9acd-e38516f7ba42 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:24.551134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13be7b-b525-4e1d-8ac9-93614e3a0cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:24.551857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13be7b-b525-4e1d-8ac9-93614e3a0cdf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:24.552142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13be7b-b525-4e1d-8ac9-93614e3a0cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:24.552424 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13be7b-b525-4e1d-8ac9-93614e3a0cdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:24.564116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13be7b-b525-4e1d-8ac9-93614e3a0cdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-ad13be7b-b525-4e1d-8ac9-93614e3a0cdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:24.571871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:24.572759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:24.573781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 417/845] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:24.580817 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b7c5d-bdde-420b-a7b2-24e2bdc8a168 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:24.651371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e071385-7723-4808-9d28-6970623af360 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:24.651734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e071385-7723-4808-9d28-6970623af360 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:24.651933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e071385-7723-4808-9d28-6970623af360 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:24.653492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 418/846] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:52:24.658606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebe6ca-6645-4bda-b40f-e2c584babc28 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:24.658893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebe6ca-6645-4bda-b40f-e2c584babc28 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:24.659174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebe6ca-6645-4bda-b40f-e2c584babc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:24.677696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-10ebe6ca-6645-4bda-b40f-e2c584babc28 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:24.679041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-10ebe6ca-6645-4bda-b40f-e2c584babc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:24.688679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b7c5d-bdde-420b-a7b2-24e2bdc8a168 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:24.711690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:24.730256 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a7b97-259e-472f-a340-5a6bdb9f6279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:24.730941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a7b97-259e-472f-a340-5a6bdb9f6279 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:24.731225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a7b97-259e-472f-a340-5a6bdb9f6279 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:24.731503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a7b97-259e-472f-a340-5a6bdb9f6279 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:24.732255 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a7b97-259e-472f-a340-5a6bdb9f6279 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:24.732565 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a7b97-259e-472f-a340-5a6bdb9f6279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:24.733939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a7b97-259e-472f-a340-5a6bdb9f6279 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:24.746901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 429/847] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:24.746901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76fd41-8592-446d-b74a-81f02940ba80 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:24.746901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76fd41-8592-446d-b74a-81f02940ba80 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:24.746901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76fd41-8592-446d-b74a-81f02940ba80 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:24.746901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 430/848] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:24.750583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76fd41-8592-446d-b74a-81f02940ba80 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:24.757505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 419/849] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:24.759809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239a9f5-8719-4174-89c7-5f8ce4f297f8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:24.759809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239a9f5-8719-4174-89c7-5f8ce4f297f8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:24.759809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239a9f5-8719-4174-89c7-5f8ce4f297f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:24.776824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6239a9f5-8719-4174-89c7-5f8ce4f297f8 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:24.778133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6239a9f5-8719-4174-89c7-5f8ce4f297f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:24.821858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:24.847135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 420/850] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:24.847947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239a9f5-8719-4174-89c7-5f8ce4f297f8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:24.868475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76fd41-8592-446d-b74a-81f02940ba80 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:24.903948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fee408-cbe8-49a2-8b94-f22d2afa76d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:24.904923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fee408-cbe8-49a2-8b94-f22d2afa76d2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:24.905274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fee408-cbe8-49a2-8b94-f22d2afa76d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:24.905616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fee408-cbe8-49a2-8b94-f22d2afa76d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:24.917382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fee408-cbe8-49a2-8b94-f22d2afa76d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:24.918989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fee408-cbe8-49a2-8b94-f22d2afa76d2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-16fee408-cbe8-49a2-8b94-f22d2afa76d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:24.936964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:24.937884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:24.938803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:24.938803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:24.938803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:24.938803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:24.938803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 431/851] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:24.943478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fee408-cbe8-49a2-8b94-f22d2afa76d2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:24.953464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:24.961006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239a9f5-8719-4174-89c7-5f8ce4f297f8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:24.996819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799de99-5b42-4c52-a4eb-4061f49c2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:24.997483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799de99-5b42-4c52-a4eb-4061f49c2b0d None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:24.997774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799de99-5b42-4c52-a4eb-4061f49c2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:24.998048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799de99-5b42-4c52-a4eb-4061f49c2b0d None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:24.998804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799de99-5b42-4c52-a4eb-4061f49c2b0d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:25.000206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799de99-5b42-4c52-a4eb-4061f49c2b0d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:25.014284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 421/852] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:52:24 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:25.028520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799de99-5b42-4c52-a4eb-4061f49c2b0d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:25.055865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fee408-cbe8-49a2-8b94-f22d2afa76d2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:25.090612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:25.118312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7a2e4-16c6-4fc8-92b7-6400d95ac101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:25.119454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7a2e4-16c6-4fc8-92b7-6400d95ac101 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:25.119922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7a2e4-16c6-4fc8-92b7-6400d95ac101 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:25.120395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7a2e4-16c6-4fc8-92b7-6400d95ac101 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:25.133040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7a2e4-16c6-4fc8-92b7-6400d95ac101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-4ce7a2e4-16c6-4fc8-92b7-6400d95ac101 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:25.141494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:25.142701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:52:25.143708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:25.146248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 432/853] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:52:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:25.162114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7a2e4-16c6-4fc8-92b7-6400d95ac101 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:25.177184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799de99-5b42-4c52-a4eb-4061f49c2b0d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:25.231046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630aa2a-4622-42aa-91a7-cf43916a203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:25.231722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630aa2a-4622-42aa-91a7-cf43916a203e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:25.232001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630aa2a-4622-42aa-91a7-cf43916a203e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:25.232278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630aa2a-4622-42aa-91a7-cf43916a203e None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:25.233008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630aa2a-4622-42aa-91a7-cf43916a203e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:25.233314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630aa2a-4622-42aa-91a7-cf43916a203e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:25.234727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630aa2a-4622-42aa-91a7-cf43916a203e None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:25.248369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 422/854] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:25.253870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630aa2a-4622-42aa-91a7-cf43916a203e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:25.274436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:25.293252 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7a2e4-16c6-4fc8-92b7-6400d95ac101 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:25.340033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb386e7-3723-48a3-aae9-a1be7218e1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:25.341506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb386e7-3723-48a3-aae9-a1be7218e1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:25.342127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb386e7-3723-48a3-aae9-a1be7218e1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:25.342710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb386e7-3723-48a3-aae9-a1be7218e1f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:25.343642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb386e7-3723-48a3-aae9-a1be7218e1f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:25.344396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb386e7-3723-48a3-aae9-a1be7218e1f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:25.346782 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb386e7-3723-48a3-aae9-a1be7218e1f9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:25.362260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 433/855] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:52:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:25.364877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb386e7-3723-48a3-aae9-a1be7218e1f9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:25.369762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630aa2a-4622-42aa-91a7-cf43916a203e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:25.405969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934859f3-97f4-45e6-a80a-ceeddb873997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:25.406808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934859f3-97f4-45e6-a80a-ceeddb873997 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:25.407100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934859f3-97f4-45e6-a80a-ceeddb873997 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:25.407381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934859f3-97f4-45e6-a80a-ceeddb873997 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:25.409624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:25.416258 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934859f3-97f4-45e6-a80a-ceeddb873997 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:25.418218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934859f3-97f4-45e6-a80a-ceeddb873997 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-934859f3-97f4-45e6-a80a-ceeddb873997 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:25.432847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:25.433907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:25.434870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:25.434870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:25.434870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:25.434870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:25.434870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:25.434870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:25.434870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:25.434870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:25.434870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 10:52:25.434870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:25.439199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 423/856] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:25.440051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934859f3-97f4-45e6-a80a-ceeddb873997 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:25.504133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb386e7-3723-48a3-aae9-a1be7218e1f9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:25.526128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:25.552336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1dcbde-b498-4f13-a026-0996d91fcdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:25.553610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dcbde-b498-4f13-a026-0996d91fcdfb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:25.554040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dcbde-b498-4f13-a026-0996d91fcdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:25.554883 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dcbde-b498-4f13-a026-0996d91fcdfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:25.557556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934859f3-97f4-45e6-a80a-ceeddb873997 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:25.570709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1dcbde-b498-4f13-a026-0996d91fcdfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:25.574077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1dcbde-b498-4f13-a026-0996d91fcdfb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-fb1dcbde-b498-4f13-a026-0996d91fcdfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:25.595674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:25.597213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:25.598379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:25.598379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:25.598379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:25.598379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:25.602228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 434/857] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:25.604280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b65aaf-c4c8-4a80-81a8-fbf960ad5e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:25.606213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1dcbde-b498-4f13-a026-0996d91fcdfb None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:25.607086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b65aaf-c4c8-4a80-81a8-fbf960ad5e9f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:25.607576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b65aaf-c4c8-4a80-81a8-fbf960ad5e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:25.608098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b65aaf-c4c8-4a80-81a8-fbf960ad5e9f None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:25.609330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b65aaf-c4c8-4a80-81a8-fbf960ad5e9f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:25.609854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b65aaf-c4c8-4a80-81a8-fbf960ad5e9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:25.611623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b65aaf-c4c8-4a80-81a8-fbf960ad5e9f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:25.632337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 424/858] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Jun 20 10:52:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:25.640966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b65aaf-c4c8-4a80-81a8-fbf960ad5e9f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:25.724694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:25.746319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1dcbde-b498-4f13-a026-0996d91fcdfb None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:25.755679 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b65aaf-c4c8-4a80-81a8-fbf960ad5e9f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:25.789168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762be432-558b-4cdb-9da4-3caf6778e8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:25.789930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be432-558b-4cdb-9da4-3caf6778e8aa None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:25.790256 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be432-558b-4cdb-9da4-3caf6778e8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:25.790513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be432-558b-4cdb-9da4-3caf6778e8aa None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:25.791234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762be432-558b-4cdb-9da4-3caf6778e8aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:25.791538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762be432-558b-4cdb-9da4-3caf6778e8aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:25.793791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762be432-558b-4cdb-9da4-3caf6778e8aa None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:25.794663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4e15-3dbc-4f84-bdcb-a0ce69321319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:25.795699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4e15-3dbc-4f84-bdcb-a0ce69321319 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:25.796289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4e15-3dbc-4f84-bdcb-a0ce69321319 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:25.796757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be4e15-3dbc-4f84-bdcb-a0ce69321319 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:25.799881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:25.815873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 435/859] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:25.817297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762be432-558b-4cdb-9da4-3caf6778e8aa None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:25.884984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4e15-3dbc-4f84-bdcb-a0ce69321319 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:25.886483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be4e15-3dbc-4f84-bdcb-a0ce69321319 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:25.927705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762be432-558b-4cdb-9da4-3caf6778e8aa None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:25.969990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64368a21-a323-4776-82d5-f5fb81611e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:25.971001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64368a21-a323-4776-82d5-f5fb81611e3f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:25.971291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64368a21-a323-4776-82d5-f5fb81611e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:25.971568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64368a21-a323-4776-82d5-f5fb81611e3f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:25.980600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64368a21-a323-4776-82d5-f5fb81611e3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:25.982424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64368a21-a323-4776-82d5-f5fb81611e3f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-64368a21-a323-4776-82d5-f5fb81611e3f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:26.001623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:26.002833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:26.003772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:26.003772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:26.003772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:26.003772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:26.003772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:26.003772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:26.003772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:26.003772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:26.003772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:26.003772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:26.007832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 436/860] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jun 20 10:52:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:26.008664 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64368a21-a323-4776-82d5-f5fb81611e3f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:26.011319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 425/861] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:25 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:26.012556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:26.019226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4e15-3dbc-4f84-bdcb-a0ce69321319 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:26.069470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:26.124105 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64368a21-a323-4776-82d5-f5fb81611e3f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:26.136958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be4e15-3dbc-4f84-bdcb-a0ce69321319 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:26.160298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4b5b0-2e75-4277-bde0-4be91cbcb494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:26.160977 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4b5b0-2e75-4277-bde0-4be91cbcb494 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:26.161260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4b5b0-2e75-4277-bde0-4be91cbcb494 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:26.161537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4b5b0-2e75-4277-bde0-4be91cbcb494 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:26.162414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4b5b0-2e75-4277-bde0-4be91cbcb494 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:26.162726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4b5b0-2e75-4277-bde0-4be91cbcb494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:26.164168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4b5b0-2e75-4277-bde0-4be91cbcb494 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:26.170669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78cc85-1343-4694-9a4b-15864e95ea99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:26.171472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78cc85-1343-4694-9a4b-15864e95ea99 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:26.171756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78cc85-1343-4694-9a4b-15864e95ea99 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:26.172032 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78cc85-1343-4694-9a4b-15864e95ea99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:26.182267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78cc85-1343-4694-9a4b-15864e95ea99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:26.187904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 437/862] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jun 20 10:52:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:26.188054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78cc85-1343-4694-9a4b-15864e95ea99 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:26.188133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e32ef-ec33-4a60-a238-60b08ed775e2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:26.188133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e32ef-ec33-4a60-a238-60b08ed775e2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:26.188133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e32ef-ec33-4a60-a238-60b08ed775e2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:26.188133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 438/863] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:26.188574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e32ef-ec33-4a60-a238-60b08ed775e2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2c78cc85-1343-4694-9a4b-15864e95ea99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:26.209209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:26.210265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:26.211132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:26.211132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:26.211132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:26.211132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:26.211132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 426/864] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:26.211975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfffcaf-2bcc-49a4-b54c-d5241204635c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:26.212258 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfffcaf-2bcc-49a4-b54c-d5241204635c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:26.212553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfffcaf-2bcc-49a4-b54c-d5241204635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:26.228762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3bfffcaf-2bcc-49a4-b54c-d5241204635c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:26.230192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3bfffcaf-2bcc-49a4-b54c-d5241204635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:26.259165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:26.303335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e32ef-ec33-4a60-a238-60b08ed775e2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:26.306378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 427/865] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:26 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:26.310332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfffcaf-2bcc-49a4-b54c-d5241204635c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:26.334278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:26.339058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6649c39-c6e9-4291-8d7a-d8a37c7227d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:26.339713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6649c39-c6e9-4291-8d7a-d8a37c7227d4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:26.340067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6649c39-c6e9-4291-8d7a-d8a37c7227d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:26.340267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6649c39-c6e9-4291-8d7a-d8a37c7227d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:26.411644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6649c39-c6e9-4291-8d7a-d8a37c7227d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:26.413039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6649c39-c6e9-4291-8d7a-d8a37c7227d4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:26.421927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfffcaf-2bcc-49a4-b54c-d5241204635c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:26.457604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592816d6-168a-421f-a40e-159a09489d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:26.458305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592816d6-168a-421f-a40e-159a09489d66 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:26.458595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592816d6-168a-421f-a40e-159a09489d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:26.458873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592816d6-168a-421f-a40e-159a09489d66 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:26.459607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592816d6-168a-421f-a40e-159a09489d66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:26.459938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592816d6-168a-421f-a40e-159a09489d66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:26.461322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592816d6-168a-421f-a40e-159a09489d66 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:26.472702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 428/866] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:26.473479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592816d6-168a-421f-a40e-159a09489d66 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:26.480975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:26.530180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 439/867] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:26 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:26.530180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6649c39-c6e9-4291-8d7a-d8a37c7227d4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:26.558316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:26.583204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592816d6-168a-421f-a40e-159a09489d66 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:26.621621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4b6d4-9205-4da1-9f2f-4631d669d727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:26.622332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b6d4-9205-4da1-9f2f-4631d669d727 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:26.622615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b6d4-9205-4da1-9f2f-4631d669d727 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:26.622894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4b6d4-9205-4da1-9f2f-4631d669d727 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:26.623622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4b6d4-9205-4da1-9f2f-4631d669d727 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:26.625004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4b6d4-9205-4da1-9f2f-4631d669d727 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:26.637356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6649c39-c6e9-4291-8d7a-d8a37c7227d4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:26.640798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 429/868] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:52:26 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:26.640798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4b6d4-9205-4da1-9f2f-4631d669d727 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:26.693321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c7c8-00a1-4fdd-af76-265b6e06c99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:26.694074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c7c8-00a1-4fdd-af76-265b6e06c99d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:26.694382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c7c8-00a1-4fdd-af76-265b6e06c99d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:26.694653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c7c8-00a1-4fdd-af76-265b6e06c99d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:26.705044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229c7c8-00a1-4fdd-af76-265b6e06c99d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:26.707180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229c7c8-00a1-4fdd-af76-265b6e06c99d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e229c7c8-00a1-4fdd-af76-265b6e06c99d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:26.726926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:26.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:26.728754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:26.728754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:26.728754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:26.728754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:26.728754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:26.728754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:26.728754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:26.728754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:26.729659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 440/869] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:26.739201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c7c8-00a1-4fdd-af76-265b6e06c99d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:26.760945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:26.784682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4b6d4-9205-4da1-9f2f-4631d669d727 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:26.836058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d923f-00ad-41b4-a1d1-99863ffd38cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:26.837030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d923f-00ad-41b4-a1d1-99863ffd38cc None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:26.837314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d923f-00ad-41b4-a1d1-99863ffd38cc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:26.837591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d923f-00ad-41b4-a1d1-99863ffd38cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:26.841045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:26.866662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c7c8-00a1-4fdd-af76-265b6e06c99d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:26.906417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d871c-8486-4eb7-81e7-415ce2dc206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:26.907298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d871c-8486-4eb7-81e7-415ce2dc206d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:26.907577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d871c-8486-4eb7-81e7-415ce2dc206d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:26.907966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d871c-8486-4eb7-81e7-415ce2dc206d None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:26.908796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d871c-8486-4eb7-81e7-415ce2dc206d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:26.909261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d871c-8486-4eb7-81e7-415ce2dc206d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:26.910796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d871c-8486-4eb7-81e7-415ce2dc206d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:26.925188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 441/870] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:26.933189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d871c-8486-4eb7-81e7-415ce2dc206d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:26.984789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d923f-00ad-41b4-a1d1-99863ffd38cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:26.986801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d923f-00ad-41b4-a1d1-99863ffd38cc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-332d923f-00ad-41b4-a1d1-99863ffd38cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:27.010058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:27.011301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:27.012186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:27.012186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:27.012186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:27.012186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:27.012186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 430/871] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:27.018053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d923f-00ad-41b4-a1d1-99863ffd38cc None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:27.030727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:27.062992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d871c-8486-4eb7-81e7-415ce2dc206d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:27.111397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230c3f1-be67-4d85-9f47-8ef2b227f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:27.112527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230c3f1-be67-4d85-9f47-8ef2b227f68d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:27.113341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230c3f1-be67-4d85-9f47-8ef2b227f68d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:27.113767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230c3f1-be67-4d85-9f47-8ef2b227f68d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:27.114796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230c3f1-be67-4d85-9f47-8ef2b227f68d None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=3c133005ca8d4478be8ff4acad5e88cc) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:27.115276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230c3f1-be67-4d85-9f47-8ef2b227f68d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:27.117613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230c3f1-be67-4d85-9f47-8ef2b227f68d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:27.126442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 442/872] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:26 2022] GET /identity/v3/role_assignments?user.id=3c133005ca8d4478be8ff4acad5e88cc&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:27.133093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d923f-00ad-41b4-a1d1-99863ffd38cc None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:27.174067 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4747f48-efaa-4eb7-8439-8a71c6a606e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:27.175041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4747f48-efaa-4eb7-8439-8a71c6a606e8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:27.175500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4747f48-efaa-4eb7-8439-8a71c6a606e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:27.175923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4747f48-efaa-4eb7-8439-8a71c6a606e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:27.176857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4747f48-efaa-4eb7-8439-8a71c6a606e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:27.177364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4747f48-efaa-4eb7-8439-8a71c6a606e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:27.178979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4747f48-efaa-4eb7-8439-8a71c6a606e8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:27.186413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38cfdd-c578-414c-9569-b27405358a0a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:27.186980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38cfdd-c578-414c-9569-b27405358a0a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:27.187412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38cfdd-c578-414c-9569-b27405358a0a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:27.189326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 443/873] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:52:27.195149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 431/874] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:27.196234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a8942-8f8d-4b5d-96a6-5f3c98ed8dd3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:27.196725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a8942-8f8d-4b5d-96a6-5f3c98ed8dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:27.197133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a8942-8f8d-4b5d-96a6-5f3c98ed8dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:27.203201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4747f48-efaa-4eb7-8439-8a71c6a606e8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:27.216181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-069a8942-8f8d-4b5d-96a6-5f3c98ed8dd3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:27.217423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-069a8942-8f8d-4b5d-96a6-5f3c98ed8dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:27.260223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:27.294545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 444/875] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:27.317749 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a8942-8f8d-4b5d-96a6-5f3c98ed8dd3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:27.322266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4747f48-efaa-4eb7-8439-8a71c6a606e8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:27.363325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2134f-cfa5-4f29-899b-7b5847feb51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:27.364133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2134f-cfa5-4f29-899b-7b5847feb51d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:27.364416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2134f-cfa5-4f29-899b-7b5847feb51d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:27.365047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2134f-cfa5-4f29-899b-7b5847feb51d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:27.375568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2134f-cfa5-4f29-899b-7b5847feb51d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:27.378387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2134f-cfa5-4f29-899b-7b5847feb51d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2ae2134f-cfa5-4f29-899b-7b5847feb51d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:27.394894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:27.395910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:27.397871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:27.397871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:27.397871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:27.397871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:27.397871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:27.397871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:27.397871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:27.397871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:27.397871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 432/876] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:27.401798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2134f-cfa5-4f29-899b-7b5847feb51d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:27.406438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:27.434917 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a8942-8f8d-4b5d-96a6-5f3c98ed8dd3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:27.472116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2004e07-a8bb-4c0f-9fdd-4c2a4a550377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:27.472937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2004e07-a8bb-4c0f-9fdd-4c2a4a550377 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:27.473262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2004e07-a8bb-4c0f-9fdd-4c2a4a550377 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:27.473534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2004e07-a8bb-4c0f-9fdd-4c2a4a550377 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:27.474219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2004e07-a8bb-4c0f-9fdd-4c2a4a550377 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:27.475591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2004e07-a8bb-4c0f-9fdd-4c2a4a550377 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-b2004e07-a8bb-4c0f-9fdd-4c2a4a550377 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:27.487920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:27.488884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:27.489735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 445/877] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Jun 20 10:52:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:27.497143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2004e07-a8bb-4c0f-9fdd-4c2a4a550377 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:27.521190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2134f-cfa5-4f29-899b-7b5847feb51d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:27.531078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:27.562516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b1036-2cc6-4cb5-8d6e-23c0d556baf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:27.563192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b1036-2cc6-4cb5-8d6e-23c0d556baf3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:27.563471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b1036-2cc6-4cb5-8d6e-23c0d556baf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:27.563746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b1036-2cc6-4cb5-8d6e-23c0d556baf3 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:27.564543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b1036-2cc6-4cb5-8d6e-23c0d556baf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:27.564851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b1036-2cc6-4cb5-8d6e-23c0d556baf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:27.566341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b1036-2cc6-4cb5-8d6e-23c0d556baf3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:27.578864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 433/878] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:27.584569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b1036-2cc6-4cb5-8d6e-23c0d556baf3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:27.614051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2004e07-a8bb-4c0f-9fdd-4c2a4a550377 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:27.652952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c572ea-2e0d-4974-aad0-0c1a248f0d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:27.654464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c572ea-2e0d-4974-aad0-0c1a248f0d38 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:27.654972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c572ea-2e0d-4974-aad0-0c1a248f0d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:27.655389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c572ea-2e0d-4974-aad0-0c1a248f0d38 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:27.656372 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c572ea-2e0d-4974-aad0-0c1a248f0d38 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:27.656923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c572ea-2e0d-4974-aad0-0c1a248f0d38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:27.658520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c572ea-2e0d-4974-aad0-0c1a248f0d38 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:27.660877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:27.670580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 446/879] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:27.671834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c572ea-2e0d-4974-aad0-0c1a248f0d38 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:27.715035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b1036-2cc6-4cb5-8d6e-23c0d556baf3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:27.757231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1815e1-8cab-4c97-96e2-5b5301d38ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:27.758047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1815e1-8cab-4c97-96e2-5b5301d38ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:27.758466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1815e1-8cab-4c97-96e2-5b5301d38ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:27.758841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1815e1-8cab-4c97-96e2-5b5301d38ac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:27.759777 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1815e1-8cab-4c97-96e2-5b5301d38ac0 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=aca3b45e80764c3b8254e24d09ff0d4c) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:27.760191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1815e1-8cab-4c97-96e2-5b5301d38ac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:27.762022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1815e1-8cab-4c97-96e2-5b5301d38ac0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:27.768695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 434/880] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:27 2022] GET /identity/v3/role_assignments?user.id=aca3b45e80764c3b8254e24d09ff0d4c&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:27.789532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c572ea-2e0d-4974-aad0-0c1a248f0d38 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:27.829710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e5411-9617-4285-9383-08b8aa7aa541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:27.830660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e5411-9617-4285-9383-08b8aa7aa541 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:27.831048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e5411-9617-4285-9383-08b8aa7aa541 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:27.831471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e5411-9617-4285-9383-08b8aa7aa541 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:27.832735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e5411-9617-4285-9383-08b8aa7aa541 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:27.833129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e5411-9617-4285-9383-08b8aa7aa541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:27.834739 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e5411-9617-4285-9383-08b8aa7aa541 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:27.841458 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 447/881] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:27.847652 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1815e1-8cab-4c97-96e2-5b5301d38ac0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:27.884427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:27.973752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1815e1-8cab-4c97-96e2-5b5301d38ac0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:28.008712 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7fda1-6846-4695-9168-34b38bb178ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:28.009427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7fda1-6846-4695-9168-34b38bb178ac None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:28.009711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7fda1-6846-4695-9168-34b38bb178ac None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:28.010075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7fda1-6846-4695-9168-34b38bb178ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:28.011094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7fda1-6846-4695-9168-34b38bb178ac None admin] RBAC: Adding query filter params (interface=public, service_id=94a067a56d81402199840a67d5fa3d5b, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:28.011376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7fda1-6846-4695-9168-34b38bb178ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:28.012790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7fda1-6846-4695-9168-34b38bb178ac None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:28.018575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 435/882] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jun 20 10:52:27 2022] GET /identity/v3/endpoints?service_id=94a067a56d81402199840a67d5fa3d5b&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:28.024026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e5411-9617-4285-9383-08b8aa7aa541 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:28.110971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:28.150900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e5411-9617-4285-9383-08b8aa7aa541 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:28.184898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ae5ae-9915-4ba4-823f-88c27dbf86ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:28.185574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae5ae-9915-4ba4-823f-88c27dbf86ac None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:28.185906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae5ae-9915-4ba4-823f-88c27dbf86ac None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:28.186215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ae5ae-9915-4ba4-823f-88c27dbf86ac None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:28.186901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ae5ae-9915-4ba4-823f-88c27dbf86ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:28.188284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ae5ae-9915-4ba4-823f-88c27dbf86ac None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:28.194401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 448/883] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jun 20 10:52:28 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 10:52:28.833987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df84867-7148-4a54-aef3-2f00f745e54d None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:28.834537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df84867-7148-4a54-aef3-2f00f745e54d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:28.834613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df84867-7148-4a54-aef3-2f00f745e54d None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:28.836201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 436/884] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:28.841586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f28a72-180e-4181-9607-f9a29abd3139 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:28.842010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f28a72-180e-4181-9607-f9a29abd3139 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:28.842208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f28a72-180e-4181-9607-f9a29abd3139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:28.859164 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-69f28a72-180e-4181-9607-f9a29abd3139 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:28.860343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-69f28a72-180e-4181-9607-f9a29abd3139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:28.930486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 449/885] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:28.953678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df84867-7148-4a54-aef3-2f00f745e54d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:29.060322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df84867-7148-4a54-aef3-2f00f745e54d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:29.061948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:29.097408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1fbfb-c4a2-418b-8d34-774549603bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:29.098160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1fbfb-c4a2-418b-8d34-774549603bce None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:29.098447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1fbfb-c4a2-418b-8d34-774549603bce None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:29.098721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1fbfb-c4a2-418b-8d34-774549603bce None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:29.099385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1fbfb-c4a2-418b-8d34-774549603bce None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:29.100780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1fbfb-c4a2-418b-8d34-774549603bce None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-54c1fbfb-c4a2-418b-8d34-774549603bce None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:29.113084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:29.114127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:29.114961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 437/886] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:52:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:29.120735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f28a72-180e-4181-9607-f9a29abd3139 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:29.181644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:29.227955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f28a72-180e-4181-9607-f9a29abd3139 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:29.263488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aebc83-c195-484f-82d8-bdd16cb79aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:29.264530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebc83-c195-484f-82d8-bdd16cb79aec None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:29.265011 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebc83-c195-484f-82d8-bdd16cb79aec None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:29.265590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebc83-c195-484f-82d8-bdd16cb79aec None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:29.266516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aebc83-c195-484f-82d8-bdd16cb79aec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:29.267021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aebc83-c195-484f-82d8-bdd16cb79aec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:29.268587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aebc83-c195-484f-82d8-bdd16cb79aec None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:29.274007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 450/887] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:52:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:29.279473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1fbfb-c4a2-418b-8d34-774549603bce None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:29.283523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57107c73-299e-43f2-8e46-7099371fdc4a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:29.283811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57107c73-299e-43f2-8e46-7099371fdc4a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:29.284091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57107c73-299e-43f2-8e46-7099371fdc4a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:29.285537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 451/888] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:29.292385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ec882-7a51-4c78-8448-a01b1ae6c29e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:29.293267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ec882-7a51-4c78-8448-a01b1ae6c29e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:29.294037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ec882-7a51-4c78-8448-a01b1ae6c29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:29.315616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8a5ec882-7a51-4c78-8448-a01b1ae6c29e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:29.316791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8a5ec882-7a51-4c78-8448-a01b1ae6c29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:29.389943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1fbfb-c4a2-418b-8d34-774549603bce None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:29.405608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 452/889] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:29.421312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ec882-7a51-4c78-8448-a01b1ae6c29e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:29.441781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e8fc7-133a-42c6-8526-b7739331f0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:29.442749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e8fc7-133a-42c6-8526-b7739331f0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:29.442936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e8fc7-133a-42c6-8526-b7739331f0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:29.443296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e8fc7-133a-42c6-8526-b7739331f0c8 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:29.444111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e8fc7-133a-42c6-8526-b7739331f0c8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:29.444509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e8fc7-133a-42c6-8526-b7739331f0c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:29.445961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e8fc7-133a-42c6-8526-b7739331f0c8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:29.452391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 438/890] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:52:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:29.458490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e8fc7-133a-42c6-8526-b7739331f0c8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:29.486268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:29.487189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:29.536798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ec882-7a51-4c78-8448-a01b1ae6c29e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:29.566668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e8fc7-133a-42c6-8526-b7739331f0c8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:29.573670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3001b-554b-44d8-baea-adbda86a24fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:29.575241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3001b-554b-44d8-baea-adbda86a24fe None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:29.575598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3001b-554b-44d8-baea-adbda86a24fe None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:29.575946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3001b-554b-44d8-baea-adbda86a24fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:29.576688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3001b-554b-44d8-baea-adbda86a24fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:29.578630 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3001b-554b-44d8-baea-adbda86a24fe None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-c4c3001b-554b-44d8-baea-adbda86a24fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:29.597091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:29.598122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:29.598971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:29.602241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 453/891] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:29.605468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3001b-554b-44d8-baea-adbda86a24fe None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:29.611379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf05a16-407f-4bb3-a8b2-875f002d9cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:29.612269 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf05a16-407f-4bb3-a8b2-875f002d9cbb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:29.612728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf05a16-407f-4bb3-a8b2-875f002d9cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:29.613178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf05a16-407f-4bb3-a8b2-875f002d9cbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:29.614376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf05a16-407f-4bb3-a8b2-875f002d9cbb None admin] RBAC: Adding query filter params (interface=public, service_id=63b23bcdafcf4c5088b2e57676e6a4a8, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:29.614868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf05a16-407f-4bb3-a8b2-875f002d9cbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:29.616984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf05a16-407f-4bb3-a8b2-875f002d9cbb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:29.624853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 439/892] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jun 20 10:52:29 2022] GET /identity/v3/endpoints?service_id=63b23bcdafcf4c5088b2e57676e6a4a8&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:29.630662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf05a16-407f-4bb3-a8b2-875f002d9cbb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:29.714540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3001b-554b-44d8-baea-adbda86a24fe None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:29.732274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:29.747332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:29.750894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa31df7-d3da-4c2a-a050-bdddc570e5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:29.751865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31df7-d3da-4c2a-a050-bdddc570e5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:29.752197 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31df7-d3da-4c2a-a050-bdddc570e5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:29.752474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31df7-d3da-4c2a-a050-bdddc570e5d6 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:29.753204 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa31df7-d3da-4c2a-a050-bdddc570e5d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:29.754316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa31df7-d3da-4c2a-a050-bdddc570e5d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:29.755996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa31df7-d3da-4c2a-a050-bdddc570e5d6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:29.765997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 454/893] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jun 20 10:52:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:29.767405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa31df7-d3da-4c2a-a050-bdddc570e5d6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:29.770822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf05a16-407f-4bb3-a8b2-875f002d9cbb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:29.814724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f4da2-a28a-41c3-a0c9-e8944c98edc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:29.815412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f4da2-a28a-41c3-a0c9-e8944c98edc1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:29.815697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f4da2-a28a-41c3-a0c9-e8944c98edc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:29.815977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f4da2-a28a-41c3-a0c9-e8944c98edc1 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:29.816679 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f4da2-a28a-41c3-a0c9-e8944c98edc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:29.818069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f4da2-a28a-41c3-a0c9-e8944c98edc1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:29.826123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 440/894] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jun 20 10:52:29 2022] GET /identity/v3/services => generated 1268 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 10:52:29.893557 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa31df7-d3da-4c2a-a050-bdddc570e5d6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:29.932377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bb882-89a5-4871-b2e8-dcb815029159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:29.933032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bb882-89a5-4871-b2e8-dcb815029159 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:29.933318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bb882-89a5-4871-b2e8-dcb815029159 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:29.933594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bb882-89a5-4871-b2e8-dcb815029159 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:29.934353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bb882-89a5-4871-b2e8-dcb815029159 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:29.934668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bb882-89a5-4871-b2e8-dcb815029159 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:29.936075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bb882-89a5-4871-b2e8-dcb815029159 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:29.941778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 455/895] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jun 20 10:52:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:29.999805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65889e-e303-4056-a51d-f37f02ec905e None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.000347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65889e-e303-4056-a51d-f37f02ec905e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.000501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65889e-e303-4056-a51d-f37f02ec905e None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.002416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 441/896] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:30.008362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131906f3-77d1-44d2-a2c0-ae785dcff944 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.008940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131906f3-77d1-44d2-a2c0-ae785dcff944 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.009398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131906f3-77d1-44d2-a2c0-ae785dcff944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.029041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-131906f3-77d1-44d2-a2c0-ae785dcff944 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:30.032020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-131906f3-77d1-44d2-a2c0-ae785dcff944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:30.077242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:30.108714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 456/897] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:30.137376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65889e-e303-4056-a51d-f37f02ec905e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:30.243547 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65889e-e303-4056-a51d-f37f02ec905e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:30.279085 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70f4f5-b86f-4d43-9988-b5a594d9e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:30.280252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f4f5-b86f-4d43-9988-b5a594d9e4bb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.280738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f4f5-b86f-4d43-9988-b5a594d9e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.281212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f4f5-b86f-4d43-9988-b5a594d9e4bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.291375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a7911-00e1-4294-be7f-af7e2ab37518 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.291834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a7911-00e1-4294-be7f-af7e2ab37518 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.291988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a7911-00e1-4294-be7f-af7e2ab37518 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.294325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 457/898] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:30.294660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70f4f5-b86f-4d43-9988-b5a594d9e4bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:30.299476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c189c1-c915-48c1-b36f-b23417241352 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.299760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c189c1-c915-48c1-b36f-b23417241352 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.300037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c189c1-c915-48c1-b36f-b23417241352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-9f70f4f5-b86f-4d43-9988-b5a594d9e4bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:30.303762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:30.304683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:30.305514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:30.307694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 442/899] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jun 20 10:52:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:30.313835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70f4f5-b86f-4d43-9988-b5a594d9e4bb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:30.321016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-22c189c1-c915-48c1-b36f-b23417241352 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:30.323102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-22c189c1-c915-48c1-b36f-b23417241352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:30.329433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:30.418274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 458/900] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:30.435452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70f4f5-b86f-4d43-9988-b5a594d9e4bb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:30.442993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c189c1-c915-48c1-b36f-b23417241352 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:30.473742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be9e53-078e-413b-91e2-c25ebf704e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:30.474545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be9e53-078e-413b-91e2-c25ebf704e52 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.474827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be9e53-078e-413b-91e2-c25ebf704e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.475102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be9e53-078e-413b-91e2-c25ebf704e52 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.476079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be9e53-078e-413b-91e2-c25ebf704e52 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:30.476389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be9e53-078e-413b-91e2-c25ebf704e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:30.477774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be9e53-078e-413b-91e2-c25ebf704e52 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:30.487399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 443/901] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jun 20 10:52:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:30.492880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be9e53-078e-413b-91e2-c25ebf704e52 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:30.567624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c189c1-c915-48c1-b36f-b23417241352 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:30.571568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:30.582634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:30.609359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4877fb9-d826-4e8a-827a-b25a4bdd3e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:30.610214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4877fb9-d826-4e8a-827a-b25a4bdd3e1a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.610581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4877fb9-d826-4e8a-827a-b25a4bdd3e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.610769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4877fb9-d826-4e8a-827a-b25a4bdd3e1a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.611428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4877fb9-d826-4e8a-827a-b25a4bdd3e1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:30.612819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4877fb9-d826-4e8a-827a-b25a4bdd3e1a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:30.618885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be9e53-078e-413b-91e2-c25ebf704e52 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a4877fb9-d826-4e8a-827a-b25a4bdd3e1a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:30.626852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:30.627800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:30.628642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 459/902] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Jun 20 10:52:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:30.633994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4877fb9-d826-4e8a-827a-b25a4bdd3e1a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:30.672959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93a477-e7da-46cb-a40a-182a00c3c8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:30.674335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93a477-e7da-46cb-a40a-182a00c3c8ec None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.674815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93a477-e7da-46cb-a40a-182a00c3c8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.675302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93a477-e7da-46cb-a40a-182a00c3c8ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.691238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93a477-e7da-46cb-a40a-182a00c3c8ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:30.693828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93a477-e7da-46cb-a40a-182a00c3c8ec None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-1f93a477-e7da-46cb-a40a-182a00c3c8ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:30.719231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:30.720343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:30.721413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:30.721413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:30.721413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:30.721413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:30.724230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 444/903] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:30.729939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93a477-e7da-46cb-a40a-182a00c3c8ec None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:30.749465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4877fb9-d826-4e8a-827a-b25a4bdd3e1a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:30.796239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18762352-c835-4eae-9b3a-023cd3e6a8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:30.797190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18762352-c835-4eae-9b3a-023cd3e6a8aa None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.797625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18762352-c835-4eae-9b3a-023cd3e6a8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.798050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18762352-c835-4eae-9b3a-023cd3e6a8aa None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.803011 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18762352-c835-4eae-9b3a-023cd3e6a8aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:30.803011 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18762352-c835-4eae-9b3a-023cd3e6a8aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:30.804438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18762352-c835-4eae-9b3a-023cd3e6a8aa None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:30.811982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 460/904] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:30.819760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18762352-c835-4eae-9b3a-023cd3e6a8aa None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:30.845147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93a477-e7da-46cb-a40a-182a00c3c8ec None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:30.853219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:30.871657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:30.881637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603d192-3ae5-47a3-b728-42aa5e6abcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:30.882390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603d192-3ae5-47a3-b728-42aa5e6abcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.882676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603d192-3ae5-47a3-b728-42aa5e6abcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.883164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603d192-3ae5-47a3-b728-42aa5e6abcb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.883678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603d192-3ae5-47a3-b728-42aa5e6abcb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:30.883989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603d192-3ae5-47a3-b728-42aa5e6abcb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:30.885434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603d192-3ae5-47a3-b728-42aa5e6abcb7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:30.902408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 445/905] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:30.904420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603d192-3ae5-47a3-b728-42aa5e6abcb7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:30.943822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18762352-c835-4eae-9b3a-023cd3e6a8aa None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:30.980788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c88123-b77b-4a1f-8e44-af704ccefe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:30.981851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c88123-b77b-4a1f-8e44-af704ccefe34 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:30.982233 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c88123-b77b-4a1f-8e44-af704ccefe34 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:30.982581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c88123-b77b-4a1f-8e44-af704ccefe34 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:30.983367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c88123-b77b-4a1f-8e44-af704ccefe34 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:30.983785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c88123-b77b-4a1f-8e44-af704ccefe34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:30.985268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c88123-b77b-4a1f-8e44-af704ccefe34 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:30.995411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 461/906] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:30.996391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c88123-b77b-4a1f-8e44-af704ccefe34 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:31.025124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603d192-3ae5-47a3-b728-42aa5e6abcb7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:31.071599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183916a-ced8-49cf-8d02-5f61ae3187f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:31.073711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183916a-ced8-49cf-8d02-5f61ae3187f1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:31.074181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183916a-ced8-49cf-8d02-5f61ae3187f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:31.074620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183916a-ced8-49cf-8d02-5f61ae3187f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:31.084465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183916a-ced8-49cf-8d02-5f61ae3187f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:31.086819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183916a-ced8-49cf-8d02-5f61ae3187f1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:31.098777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-e183916a-ced8-49cf-8d02-5f61ae3187f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:31.106187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:31.107441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:31.108668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:31.108668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:31.108668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:31.108668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:31.108668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:31.108668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:31.108668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:31.108668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:31.108668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:31.108668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:31.109381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c88123-b77b-4a1f-8e44-af704ccefe34 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:31.111188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 446/907] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jun 20 10:52:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:31.118338 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183916a-ced8-49cf-8d02-5f61ae3187f1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:31.157943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457558e-3486-4161-9297-0b3052a0091f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:31.159223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457558e-3486-4161-9297-0b3052a0091f None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:31.160083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457558e-3486-4161-9297-0b3052a0091f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:31.160560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457558e-3486-4161-9297-0b3052a0091f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:31.161734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457558e-3486-4161-9297-0b3052a0091f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:31.163594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457558e-3486-4161-9297-0b3052a0091f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:31.180043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:31.189624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 462/908] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jun 20 10:52:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:31.242487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183916a-ced8-49cf-8d02-5f61ae3187f1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:31.287852 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4e7e0-0388-4f4d-a3d1-22a618f106e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:31.288954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4e7e0-0388-4f4d-a3d1-22a618f106e3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:31.289417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4e7e0-0388-4f4d-a3d1-22a618f106e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:31.289872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4e7e0-0388-4f4d-a3d1-22a618f106e3 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:31.290937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4e7e0-0388-4f4d-a3d1-22a618f106e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:31.291440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4e7e0-0388-4f4d-a3d1-22a618f106e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:31.293384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4e7e0-0388-4f4d-a3d1-22a618f106e3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:31.313942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 447/909] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jun 20 10:52:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:31.323526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457558e-3486-4161-9297-0b3052a0091f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:31.433041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457558e-3486-4161-9297-0b3052a0091f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:31.474385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42044fe2-9830-408c-b62d-464688805a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:31.475055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42044fe2-9830-408c-b62d-464688805a8a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:31.475345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42044fe2-9830-408c-b62d-464688805a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:31.475625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42044fe2-9830-408c-b62d-464688805a8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:31.510810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:31.563308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42044fe2-9830-408c-b62d-464688805a8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:31.565107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42044fe2-9830-408c-b62d-464688805a8a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:31.672081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 463/910] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:31 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:31.678395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4e7e0-0388-4f4d-a3d1-22a618f106e3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:31.779241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:31.782774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4e7e0-0388-4f4d-a3d1-22a618f106e3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:31.826122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b0cd0-9133-469e-b7f0-8aee8391d658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:31.826943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0cd0-9133-469e-b7f0-8aee8391d658 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:31.827229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0cd0-9133-469e-b7f0-8aee8391d658 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:31.827506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0cd0-9133-469e-b7f0-8aee8391d658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:31.847691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b0cd0-9133-469e-b7f0-8aee8391d658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:31.849189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b0cd0-9133-469e-b7f0-8aee8391d658 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d98b0cd0-9133-469e-b7f0-8aee8391d658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:31.869265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:31.870402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:31.871284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:31.871284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:31.871284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:31.871284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:31.871948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 448/911] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:31.877765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42044fe2-9830-408c-b62d-464688805a8a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:31.946498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:31.977759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bb577-d440-480d-97f0-9d2e64bb5614 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:31.978128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bb577-d440-480d-97f0-9d2e64bb5614 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:31.978328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651bb577-d440-480d-97f0-9d2e64bb5614 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:31.980341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 449/912] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:31.985676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9990ddf-4bda-41d9-b1fd-30dfe7ada9ca None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:31.985830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9990ddf-4bda-41d9-b1fd-30dfe7ada9ca None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:31.986024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9990ddf-4bda-41d9-b1fd-30dfe7ada9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.002705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42044fe2-9830-408c-b62d-464688805a8a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:32.003821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e9990ddf-4bda-41d9-b1fd-30dfe7ada9ca None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:32.005092 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e9990ddf-4bda-41d9-b1fd-30dfe7ada9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:32.042540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1df22a-0aaf-4670-9be0-1c1768543813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:32.043209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1df22a-0aaf-4670-9be0-1c1768543813 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.043495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1df22a-0aaf-4670-9be0-1c1768543813 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.043773 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1df22a-0aaf-4670-9be0-1c1768543813 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.044503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1df22a-0aaf-4670-9be0-1c1768543813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:32.044813 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1df22a-0aaf-4670-9be0-1c1768543813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:32.046227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1df22a-0aaf-4670-9be0-1c1768543813 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:32.058866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 464/913] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:32.060255 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7abcc-48bc-4d99-a8a2-b68db6941ba7 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.060548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7abcc-48bc-4d99-a8a2-b68db6941ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.060825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7abcc-48bc-4d99-a8a2-b68db6941ba7 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.062336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 465/914] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:32.064339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7abcc-48bc-4d99-a8a2-b68db6941ba7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:32.087027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 450/915] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:32.096062 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fad1da-615d-43b0-87b7-cc36f808e3dc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.096354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fad1da-615d-43b0-87b7-cc36f808e3dc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.096632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fad1da-615d-43b0-87b7-cc36f808e3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.125857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-49fad1da-615d-43b0-87b7-cc36f808e3dc None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:32.127752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-49fad1da-615d-43b0-87b7-cc36f808e3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:32.175718 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7abcc-48bc-4d99-a8a2-b68db6941ba7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:32.189713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:32.206339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 451/916] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:32.206339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fad1da-615d-43b0-87b7-cc36f808e3dc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:32.220650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd2441-c79a-4f4c-b04b-607b6da7f906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:32.221404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd2441-c79a-4f4c-b04b-607b6da7f906 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.221683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd2441-c79a-4f4c-b04b-607b6da7f906 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.222221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd2441-c79a-4f4c-b04b-607b6da7f906 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.222454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:32.229797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd2441-c79a-4f4c-b04b-607b6da7f906 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:32.231393 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd2441-c79a-4f4c-b04b-607b6da7f906 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a0bd2441-c79a-4f4c-b04b-607b6da7f906 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:32.246143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:32.247048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:32.253324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:32.253324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:32.253324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:32.253324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:32.253324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:32.253324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:32.253324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:32.253324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:32.253324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 466/917] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:32.253324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd2441-c79a-4f4c-b04b-607b6da7f906 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:32.323784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fad1da-615d-43b0-87b7-cc36f808e3dc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:32.356168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd2441-c79a-4f4c-b04b-607b6da7f906 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:32.360655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e40bc0-ec02-4600-8083-34ea6628f24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:32.361493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e40bc0-ec02-4600-8083-34ea6628f24f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.361776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e40bc0-ec02-4600-8083-34ea6628f24f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.362053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e40bc0-ec02-4600-8083-34ea6628f24f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.362742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e40bc0-ec02-4600-8083-34ea6628f24f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:32.364139 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e40bc0-ec02-4600-8083-34ea6628f24f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d0e40bc0-ec02-4600-8083-34ea6628f24f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:32.376394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:32.377454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:32.378574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 452/918] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:52:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:32.379367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e40bc0-ec02-4600-8083-34ea6628f24f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:32.400097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4c21e-53e3-46c7-a34b-0a02ba455757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:32.400788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4c21e-53e3-46c7-a34b-0a02ba455757 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.401173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4c21e-53e3-46c7-a34b-0a02ba455757 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.401606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4c21e-53e3-46c7-a34b-0a02ba455757 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.402372 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4c21e-53e3-46c7-a34b-0a02ba455757 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:32.403754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4c21e-53e3-46c7-a34b-0a02ba455757 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:32.430122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 467/919] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:52:32 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:32.430122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4c21e-53e3-46c7-a34b-0a02ba455757 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:32.471545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:32.489586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:32.521826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e40bc0-ec02-4600-8083-34ea6628f24f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:32.566629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4c21e-53e3-46c7-a34b-0a02ba455757 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:32.573226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f5d3d-e821-4950-8873-3f10efa91247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:32.574054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f5d3d-e821-4950-8873-3f10efa91247 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.574426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f5d3d-e821-4950-8873-3f10efa91247 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.574763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f5d3d-e821-4950-8873-3f10efa91247 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.575642 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f5d3d-e821-4950-8873-3f10efa91247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:32.576060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f5d3d-e821-4950-8873-3f10efa91247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:32.578382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f5d3d-e821-4950-8873-3f10efa91247 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:32.596400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 453/920] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:32.602114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f5d3d-e821-4950-8873-3f10efa91247 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:32.614818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7687be-f232-47ee-99e3-fb71b01c608d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:32.615786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7687be-f232-47ee-99e3-fb71b01c608d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.616136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7687be-f232-47ee-99e3-fb71b01c608d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.616690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7687be-f232-47ee-99e3-fb71b01c608d None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.617668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7687be-f232-47ee-99e3-fb71b01c608d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:32.618177 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7687be-f232-47ee-99e3-fb71b01c608d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:32.620066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7687be-f232-47ee-99e3-fb71b01c608d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:32.631822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 468/921] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:52:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:32.632860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7687be-f232-47ee-99e3-fb71b01c608d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:32.726398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f5d3d-e821-4950-8873-3f10efa91247 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:32.749741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:32.758348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:32.763396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7687be-f232-47ee-99e3-fb71b01c608d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:32.765069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc512e5-fe99-428d-aeae-42afa3b3cdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:32.765986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc512e5-fe99-428d-aeae-42afa3b3cdc0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.766424 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc512e5-fe99-428d-aeae-42afa3b3cdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.766836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc512e5-fe99-428d-aeae-42afa3b3cdc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.767822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc512e5-fe99-428d-aeae-42afa3b3cdc0 None admin] RBAC: Adding query filter params (role.id=6db9b3d9148e4e37bded8945be0f6f6f, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=aca3b45e80764c3b8254e24d09ff0d4c) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:32.768288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc512e5-fe99-428d-aeae-42afa3b3cdc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:32.769869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc512e5-fe99-428d-aeae-42afa3b3cdc0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:32.776971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 454/922] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:32 2022] GET /identity/v3/role_assignments?user.id=aca3b45e80764c3b8254e24d09ff0d4c&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=6db9b3d9148e4e37bded8945be0f6f6f => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:32.810502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b7732-65f7-4e17-8153-6907c5464b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:32.811533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b7732-65f7-4e17-8153-6907c5464b46 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.812049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b7732-65f7-4e17-8153-6907c5464b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.812463 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b7732-65f7-4e17-8153-6907c5464b46 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.813336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b7732-65f7-4e17-8153-6907c5464b46 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:32.813786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b7732-65f7-4e17-8153-6907c5464b46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:32.815466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b7732-65f7-4e17-8153-6907c5464b46 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:32.821979 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 469/923] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Jun 20 10:52:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:32.831288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc512e5-fe99-428d-aeae-42afa3b3cdc0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:32.953545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc512e5-fe99-428d-aeae-42afa3b3cdc0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:32.991296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cd9fa-1708-4cce-9dfc-6e084146576f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:32.992102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cd9fa-1708-4cce-9dfc-6e084146576f None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:32.992485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cd9fa-1708-4cce-9dfc-6e084146576f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:32.992865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cd9fa-1708-4cce-9dfc-6e084146576f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:32.993722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cd9fa-1708-4cce-9dfc-6e084146576f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:32.995346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cd9fa-1708-4cce-9dfc-6e084146576f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:33.018760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 455/924] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jun 20 10:52:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:33.407083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c16e7-2c5c-4abe-b66a-c9a599be8483 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:33.407823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c16e7-2c5c-4abe-b66a-c9a599be8483 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:33.407823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c16e7-2c5c-4abe-b66a-c9a599be8483 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:33.409248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 470/925] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:33.414788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e69ef-59e6-40ab-bcd3-4e9820474bf5 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:33.415282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e69ef-59e6-40ab-bcd3-4e9820474bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:33.415443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e69ef-59e6-40ab-bcd3-4e9820474bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:33.433369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e83e69ef-59e6-40ab-bcd3-4e9820474bf5 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:33.434598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e83e69ef-59e6-40ab-bcd3-4e9820474bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:33.483236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:33.512261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 456/926] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:33.535031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c16e7-2c5c-4abe-b66a-c9a599be8483 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:33.641302 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c16e7-2c5c-4abe-b66a-c9a599be8483 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:33.678435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a51764-b241-4e6d-bf66-e5ded9dab4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:33.679218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a51764-b241-4e6d-bf66-e5ded9dab4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:33.679499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a51764-b241-4e6d-bf66-e5ded9dab4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:33.679776 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a51764-b241-4e6d-bf66-e5ded9dab4e4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:33.680435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a51764-b241-4e6d-bf66-e5ded9dab4e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:33.681819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a51764-b241-4e6d-bf66-e5ded9dab4e4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a3a51764-b241-4e6d-bf66-e5ded9dab4e4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:33.693543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:33.694566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:33.695534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 471/927] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:33.701558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e69ef-59e6-40ab-bcd3-4e9820474bf5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:33.810470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e69ef-59e6-40ab-bcd3-4e9820474bf5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:33.846601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d208e-4a04-4b17-af22-048f0e8f6070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:33.847282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d208e-4a04-4b17-af22-048f0e8f6070 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:33.847569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d208e-4a04-4b17-af22-048f0e8f6070 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:33.847850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d208e-4a04-4b17-af22-048f0e8f6070 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:33.848574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d208e-4a04-4b17-af22-048f0e8f6070 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:33.848889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d208e-4a04-4b17-af22-048f0e8f6070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:33.850320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d208e-4a04-4b17-af22-048f0e8f6070 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:33.855785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 457/928] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:33.861244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a51764-b241-4e6d-bf66-e5ded9dab4e4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:33.883357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:33.969530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a51764-b241-4e6d-bf66-e5ded9dab4e4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:34.002795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcf646-a5b9-4ec3-a17f-df7e6702ae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:34.003466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcf646-a5b9-4ec3-a17f-df7e6702ae04 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:34.003746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcf646-a5b9-4ec3-a17f-df7e6702ae04 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:34.004021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcf646-a5b9-4ec3-a17f-df7e6702ae04 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:34.004730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcf646-a5b9-4ec3-a17f-df7e6702ae04 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:34.005037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcf646-a5b9-4ec3-a17f-df7e6702ae04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:34.006444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bcf646-a5b9-4ec3-a17f-df7e6702ae04 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:34.011405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 472/929] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:34.624696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefad4a-d672-48e4-930a-36a40695534b None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:34.625221 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefad4a-d672-48e4-930a-36a40695534b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:34.625333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefad4a-d672-48e4-930a-36a40695534b None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:34.627108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 458/930] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:34.632904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517daab-cf02-4c80-a287-00d79493cc92 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:34.633328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517daab-cf02-4c80-a287-00d79493cc92 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:34.633504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517daab-cf02-4c80-a287-00d79493cc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:34.651063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5517daab-cf02-4c80-a287-00d79493cc92 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:34.652265 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5517daab-cf02-4c80-a287-00d79493cc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:34.726918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 473/931] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:34.750137 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefad4a-d672-48e4-930a-36a40695534b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:34.798188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:34.855820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bb40a-12a7-4e18-9423-52b788f00018 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:34.856498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bb40a-12a7-4e18-9423-52b788f00018 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:34.856970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bb40a-12a7-4e18-9423-52b788f00018 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:34.858934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 474/932] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:52:34.860810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefad4a-d672-48e4-930a-36a40695534b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:34.865628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ad690-e75a-4c8d-ae99-bebbe43d66cc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:34.866171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ad690-e75a-4c8d-ae99-bebbe43d66cc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:34.866651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ad690-e75a-4c8d-ae99-bebbe43d66cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:34.884447 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e18ad690-e75a-4c8d-ae99-bebbe43d66cc None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:34.885576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e18ad690-e75a-4c8d-ae99-bebbe43d66cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:34.898973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:34.902717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f207b29-6bbb-4dee-92d6-8667af6763bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:34.903546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f207b29-6bbb-4dee-92d6-8667af6763bd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:34.903830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f207b29-6bbb-4dee-92d6-8667af6763bd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:34.904109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f207b29-6bbb-4dee-92d6-8667af6763bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:34.904776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f207b29-6bbb-4dee-92d6-8667af6763bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:34.906363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f207b29-6bbb-4dee-92d6-8667af6763bd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-7f207b29-6bbb-4dee-92d6-8667af6763bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:34.920521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:34.921527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:34.922359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 459/933] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:34.928211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f207b29-6bbb-4dee-92d6-8667af6763bd None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:34.970228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 475/934] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:34.986821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ad690-e75a-4c8d-ae99-bebbe43d66cc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:35.034751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f207b29-6bbb-4dee-92d6-8667af6763bd None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.042726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:35.072670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b08ddb-0e11-4d40-9e9b-134bcaed47ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.073629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b08ddb-0e11-4d40-9e9b-134bcaed47ff None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.073824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b08ddb-0e11-4d40-9e9b-134bcaed47ff None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.073983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b08ddb-0e11-4d40-9e9b-134bcaed47ff None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.074710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b08ddb-0e11-4d40-9e9b-134bcaed47ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:35.075024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b08ddb-0e11-4d40-9e9b-134bcaed47ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:35.076588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b08ddb-0e11-4d40-9e9b-134bcaed47ff None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:35.087325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 460/935] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jun 20 10:52:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:35.088039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ad690-e75a-4c8d-ae99-bebbe43d66cc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.088268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b08ddb-0e11-4d40-9e9b-134bcaed47ff None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:35.123476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6c2a4-4d80-438f-be45-1445296a3de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.124305 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6c2a4-4d80-438f-be45-1445296a3de9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.124648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6c2a4-4d80-438f-be45-1445296a3de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.124929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6c2a4-4d80-438f-be45-1445296a3de9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.136932 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6c2a4-4d80-438f-be45-1445296a3de9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-dcc6c2a4-4d80-438f-be45-1445296a3de9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:35.147491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:35.148796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:35.150178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:35.154163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 476/936] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jun 20 10:52:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:35.155507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6c2a4-4d80-438f-be45-1445296a3de9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:35.159831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:35.198732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b08ddb-0e11-4d40-9e9b-134bcaed47ff None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.235167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2289be-e6f8-4cb3-b9fa-29f81ea72cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.235948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2289be-e6f8-4cb3-b9fa-29f81ea72cef None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.236236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2289be-e6f8-4cb3-b9fa-29f81ea72cef None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.236512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2289be-e6f8-4cb3-b9fa-29f81ea72cef None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.237229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2289be-e6f8-4cb3-b9fa-29f81ea72cef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:35.237542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2289be-e6f8-4cb3-b9fa-29f81ea72cef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:35.239092 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2289be-e6f8-4cb3-b9fa-29f81ea72cef None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:35.248938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 461/937] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jun 20 10:52:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:35.248938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804f6fb-a5cb-4134-80ce-59c43167b710 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.248938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804f6fb-a5cb-4134-80ce-59c43167b710 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.248938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804f6fb-a5cb-4134-80ce-59c43167b710 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.248938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 462/938] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:35.251823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804f6fb-a5cb-4134-80ce-59c43167b710 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:35.272598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6c2a4-4d80-438f-be45-1445296a3de9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.287772 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:35.310505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b674c30-da24-45a5-9e11-7cef35d7a9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.311529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b674c30-da24-45a5-9e11-7cef35d7a9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.311902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b674c30-da24-45a5-9e11-7cef35d7a9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.312178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b674c30-da24-45a5-9e11-7cef35d7a9e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.312919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b674c30-da24-45a5-9e11-7cef35d7a9e7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:35.313222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b674c30-da24-45a5-9e11-7cef35d7a9e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:35.314612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b674c30-da24-45a5-9e11-7cef35d7a9e7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:35.323308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 477/939] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jun 20 10:52:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:35.324395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c103d-5383-4439-a49c-ec97be2d84cb None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.324676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c103d-5383-4439-a49c-ec97be2d84cb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.324949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c103d-5383-4439-a49c-ec97be2d84cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.342071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-684c103d-5383-4439-a49c-ec97be2d84cb None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:35.343268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-684c103d-5383-4439-a49c-ec97be2d84cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:35.366851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804f6fb-a5cb-4134-80ce-59c43167b710 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.401441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be312cf-0f8c-4fa8-aa96-72f538a2cd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.402026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be312cf-0f8c-4fa8-aa96-72f538a2cd07 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.402337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be312cf-0f8c-4fa8-aa96-72f538a2cd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.402620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be312cf-0f8c-4fa8-aa96-72f538a2cd07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.403415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be312cf-0f8c-4fa8-aa96-72f538a2cd07 None admin] RBAC: Adding query filter params (interface=public, service_id=388f3980d2304cb9ac4022e3282da635, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:35.403728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be312cf-0f8c-4fa8-aa96-72f538a2cd07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:35.405115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be312cf-0f8c-4fa8-aa96-72f538a2cd07 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:35.406663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:35.411496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 463/940] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jun 20 10:52:35 2022] GET /identity/v3/endpoints?service_id=388f3980d2304cb9ac4022e3282da635&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:35.412293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be312cf-0f8c-4fa8-aa96-72f538a2cd07 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:35.430108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 478/941] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:35.430108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c103d-5383-4439-a49c-ec97be2d84cb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:35.526681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:35.529753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c103d-5383-4439-a49c-ec97be2d84cb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.532457 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be312cf-0f8c-4fa8-aa96-72f538a2cd07 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.572350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba31c30-6b6c-48fd-a15b-ef49e81fabcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.572733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca3e9b-3b9b-4614-9b83-079e0f833cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.573120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba31c30-6b6c-48fd-a15b-ef49e81fabcc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.573395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba31c30-6b6c-48fd-a15b-ef49e81fabcc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.573550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca3e9b-3b9b-4614-9b83-079e0f833cec None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.573757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba31c30-6b6c-48fd-a15b-ef49e81fabcc None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.573865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca3e9b-3b9b-4614-9b83-079e0f833cec None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.574082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca3e9b-3b9b-4614-9b83-079e0f833cec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.574476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba31c30-6b6c-48fd-a15b-ef49e81fabcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:35.575869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba31c30-6b6c-48fd-a15b-ef49e81fabcc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:35.590427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 479/942] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jun 20 10:52:35 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 10:52:35.590427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba31c30-6b6c-48fd-a15b-ef49e81fabcc None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:35.593932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca3e9b-3b9b-4614-9b83-079e0f833cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:35.595580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca3e9b-3b9b-4614-9b83-079e0f833cec None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0dca3e9b-3b9b-4614-9b83-079e0f833cec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:35.615860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:35.616855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:35.617727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:35.617727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:35.617727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:35.617727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:35.623201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 464/943] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:35.624717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca3e9b-3b9b-4614-9b83-079e0f833cec None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:35.667285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:35.730483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba31c30-6b6c-48fd-a15b-ef49e81fabcc None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.751141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca3e9b-3b9b-4614-9b83-079e0f833cec None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.775759 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.776489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.776834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.777129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.777820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:35.779359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:35.780805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WARNING keystone.server.flask.common [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 20 10:52:35.797682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc9923-7fa2-4d1e-8ce6-96731d56111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.798415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc9923-7fa2-4d1e-8ce6-96731d56111b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.798736 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc9923-7fa2-4d1e-8ce6-96731d56111b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.799076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc9923-7fa2-4d1e-8ce6-96731d56111b None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.799706 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.sql.core [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 20 10:52:35.799706 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 20 10:52:35.799706 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [parameters: {'id': '52f00ab2097c470280ffd905b43f4b99', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 20 10:52:35.799706 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80133) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 20 10:52:35.800011 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc9923-7fa2-4d1e-8ce6-96731d56111b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:35.800011 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc9923-7fa2-4d1e-8ce6-96731d56111b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:35.801403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc9923-7fa2-4d1e-8ce6-96731d56111b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52f00ab2097c470280ffd905b43f4b99.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52f00ab2097c470280ffd905b43f4b99. Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application conn.query(q) Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 10:52:35.819536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result.read() Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 20 10:52:35.820413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application next(self.gen) Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application yield resource Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application next(self.gen) Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.session.rollback() Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.force_reraise() Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise self.value Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application session.commit() Jun 20 10:52:35.821390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._prepare_impl() Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.session.flush() Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._flush(objects) Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application compat.raise_( Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application flush_context.execute() Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 20 10:52:35.822356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rec.execute(self) Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 20 10:52:35.823304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application conn.query(q) Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result.read() Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 10:52:35.824109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [parameters: {'id': '52f00ab2097c470280ffd905b43f4b99', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:35.824952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:35.825700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:35.828717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:35.828717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 20 10:52:35.828717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 20 10:52:35.828717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:35.828717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:35.828717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 20 10:52:35.828717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 20 10:52:35.828717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52f00ab2097c470280ffd905b43f4b99. Jun 20 10:52:35.828717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:35.828717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 480/945] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Jun 20 10:52:35 2022] POST /identity/v3/projects => generated 262 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:35.830024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 465/945] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:35.830024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc9923-7fa2-4d1e-8ce6-96731d56111b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:35.838327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:35.842675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:35.953077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d77b7-3c09-438e-9680-074c5f0f4a79 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.954847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc9923-7fa2-4d1e-8ce6-96731d56111b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:35.987023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:35.994353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f4f25-8d74-40e7-81b5-9d00d108a412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.995347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4f25-8d74-40e7-81b5-9d00d108a412 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.995692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4f25-8d74-40e7-81b5-9d00d108a412 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:35.996006 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f4f25-8d74-40e7-81b5-9d00d108a412 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:35.998563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d35c5-1ebf-419e-be4e-9cc74a77d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:35.999514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d35c5-1ebf-419e-be4e-9cc74a77d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:35.999966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d35c5-1ebf-419e-be4e-9cc74a77d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:36.000304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d35c5-1ebf-419e-be4e-9cc74a77d9a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:36.005597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f4f25-8d74-40e7-81b5-9d00d108a412 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:36.007571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f4f25-8d74-40e7-81b5-9d00d108a412 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:36.007730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d35c5-1ebf-419e-be4e-9cc74a77d9a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:36.009499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d35c5-1ebf-419e-be4e-9cc74a77d9a3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1b5f4f25-8d74-40e7-81b5-9d00d108a412 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:36.023835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:36.024802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:36.025741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:36.025741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:36.025741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:36.025741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:36.025741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:36.025741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:36.025741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:36.025741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:36.025741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:36.025741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-bb6d35c5-1ebf-419e-be4e-9cc74a77d9a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:36.026411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:36.027266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:36.028131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:36.028131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:36.028131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:36.028131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:36.028131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:36.028131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:36.028131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:36.028131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:36.038128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 466/947] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:36.038413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 481/947] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jun 20 10:52:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:36.038576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d35c5-1ebf-419e-be4e-9cc74a77d9a3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:36.038721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f4f25-8d74-40e7-81b5-9d00d108a412 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:36.107189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:36.150301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d35c5-1ebf-419e-be4e-9cc74a77d9a3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:36.153614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f4f25-8d74-40e7-81b5-9d00d108a412 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:36.185318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412610a-7780-49e1-8a70-f2c42bd41e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:36.185989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412610a-7780-49e1-8a70-f2c42bd41e41 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:36.186314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412610a-7780-49e1-8a70-f2c42bd41e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:36.186575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412610a-7780-49e1-8a70-f2c42bd41e41 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:36.187339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412610a-7780-49e1-8a70-f2c42bd41e41 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:36.187644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412610a-7780-49e1-8a70-f2c42bd41e41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:36.189106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412610a-7780-49e1-8a70-f2c42bd41e41 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:36.189991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ec84-c578-454f-b20d-57f8719c9fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:36.190692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ec84-c578-454f-b20d-57f8719c9fdb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:36.190973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ec84-c578-454f-b20d-57f8719c9fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:36.191245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ec84-c578-454f-b20d-57f8719c9fdb None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:36.192072 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21ec84-c578-454f-b20d-57f8719c9fdb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:36.192381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21ec84-c578-454f-b20d-57f8719c9fdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:36.193804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21ec84-c578-454f-b20d-57f8719c9fdb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:36.214203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 467/948] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:36 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:36.214203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa911e0-0c5c-4bc4-8c51-b224defffe52 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:36.214203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa911e0-0c5c-4bc4-8c51-b224defffe52 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:36.214203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa911e0-0c5c-4bc4-8c51-b224defffe52 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:36.214203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 468/949] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:36.217054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b972f7-2d74-43b2-851b-aa9c71613e08 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:36.217054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b972f7-2d74-43b2-851b-aa9c71613e08 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:36.217054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b972f7-2d74-43b2-851b-aa9c71613e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:36.227494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 482/950] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jun 20 10:52:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:36.228920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ec84-c578-454f-b20d-57f8719c9fdb None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:36.236214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c5b972f7-2d74-43b2-851b-aa9c71613e08 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:36.237436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c5b972f7-2d74-43b2-851b-aa9c71613e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:36.282934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:36.322450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 469/951] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:36.343489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b972f7-2d74-43b2-851b-aa9c71613e08 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:36.346071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ec84-c578-454f-b20d-57f8719c9fdb None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:36.396787 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436add67-51d7-489f-bc80-15ad271ed981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:36.397605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436add67-51d7-489f-bc80-15ad271ed981 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:36.397886 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436add67-51d7-489f-bc80-15ad271ed981 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:36.399941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436add67-51d7-489f-bc80-15ad271ed981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:36.410334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436add67-51d7-489f-bc80-15ad271ed981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:36.411898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436add67-51d7-489f-bc80-15ad271ed981 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:36.431394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-436add67-51d7-489f-bc80-15ad271ed981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:36.435911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:36.436786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:36.437641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:36.437641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:36.437641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:36.437641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:36.438346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 483/952] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:36.444401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436add67-51d7-489f-bc80-15ad271ed981 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:36.485171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b972f7-2d74-43b2-851b-aa9c71613e08 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:36.530855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d7a59-bd44-40ac-9b98-4676790d7d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:36.531682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d7a59-bd44-40ac-9b98-4676790d7d6d None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:36.531986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d7a59-bd44-40ac-9b98-4676790d7d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:36.532339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d7a59-bd44-40ac-9b98-4676790d7d6d None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:36.533486 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d7a59-bd44-40ac-9b98-4676790d7d6d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:36.534652 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d7a59-bd44-40ac-9b98-4676790d7d6d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:36.551010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 470/953] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:52:36 2022] POST /identity/v3/services => generated 242 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:36.562515 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:36.578857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436add67-51d7-489f-bc80-15ad271ed981 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:36.629678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ab424-6537-4ea7-9c45-2a70084944aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:36.630596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ab424-6537-4ea7-9c45-2a70084944aa None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:36.630955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ab424-6537-4ea7-9c45-2a70084944aa None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:36.631468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ab424-6537-4ea7-9c45-2a70084944aa None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:36.632304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ab424-6537-4ea7-9c45-2a70084944aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:36.632688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ab424-6537-4ea7-9c45-2a70084944aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:36.634267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ab424-6537-4ea7-9c45-2a70084944aa None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:36.648584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 484/954] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:36.654655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d7a59-bd44-40ac-9b98-4676790d7d6d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:36.764827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d7a59-bd44-40ac-9b98-4676790d7d6d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:36.789017 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:36.805328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfe423-93a5-44f4-b398-affd8a636956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:36.806182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfe423-93a5-44f4-b398-affd8a636956 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:36.806464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfe423-93a5-44f4-b398-affd8a636956 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:36.806735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcfe423-93a5-44f4-b398-affd8a636956 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:36.814395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfe423-93a5-44f4-b398-affd8a636956 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:36.815854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcfe423-93a5-44f4-b398-affd8a636956 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-0fcfe423-93a5-44f4-b398-affd8a636956 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:36.832228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:36.833335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:36.834287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:36.834287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:36.834287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:36.834287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:36.834287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:36.834287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:36.834287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:36.834287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:36.834287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 471/955] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:36.839709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ab424-6537-4ea7-9c45-2a70084944aa None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:36.950449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ab424-6537-4ea7-9c45-2a70084944aa None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:36.989427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427894af-5d16-4865-95f5-6128d2cf5e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:36.990116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427894af-5d16-4865-95f5-6128d2cf5e4c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:36.990418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427894af-5d16-4865-95f5-6128d2cf5e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:36.990697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427894af-5d16-4865-95f5-6128d2cf5e4c None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:36.991490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427894af-5d16-4865-95f5-6128d2cf5e4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:36.991799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427894af-5d16-4865-95f5-6128d2cf5e4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:36.993312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427894af-5d16-4865-95f5-6128d2cf5e4c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:37.006119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 485/956] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:37.012403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfe423-93a5-44f4-b398-affd8a636956 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:37.116045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcfe423-93a5-44f4-b398-affd8a636956 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:37.151638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af25d10-7f6e-42a1-a758-a9940f89f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:37.152540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25d10-7f6e-42a1-a758-a9940f89f5dc None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:37.152822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25d10-7f6e-42a1-a758-a9940f89f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:37.153100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25d10-7f6e-42a1-a758-a9940f89f5dc None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/aca3b45e80764c3b8254e24d09ff0d4c/roles/6db9b3d9148e4e37bded8945be0f6f6f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:37.155345 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af25d10-7f6e-42a1-a758-a9940f89f5dc None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=aca3b45e80764c3b8254e24d09ff0d4c, role_id=6db9b3d9148e4e37bded8945be0f6f6f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:37.156805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af25d10-7f6e-42a1-a758-a9940f89f5dc None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:37.182361 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 472/957] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:52:37 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/aca3b45e80764c3b8254e24d09ff0d4c/roles/6db9b3d9148e4e37bded8945be0f6f6f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:52:37.701946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6321776-95dc-4b3e-8f22-b5a5f0925ec2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:37.702382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6321776-95dc-4b3e-8f22-b5a5f0925ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:37.702560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6321776-95dc-4b3e-8f22-b5a5f0925ec2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:37.704170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 486/958] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:37.709686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465ca08-78c4-4815-83ca-c9023f5ab414 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:37.709969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465ca08-78c4-4815-83ca-c9023f5ab414 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:37.710284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465ca08-78c4-4815-83ca-c9023f5ab414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:37.726886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e465ca08-78c4-4815-83ca-c9023f5ab414 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:37.728057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e465ca08-78c4-4815-83ca-c9023f5ab414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:37.762745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:37.798211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 473/959] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:37.820791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6321776-95dc-4b3e-8f22-b5a5f0925ec2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:37.823872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:37.923558 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6321776-95dc-4b3e-8f22-b5a5f0925ec2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:37.957973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539dda8-8a7e-4ac2-a56a-3db2d7c259d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:37.958740 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539dda8-8a7e-4ac2-a56a-3db2d7c259d9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:37.959020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539dda8-8a7e-4ac2-a56a-3db2d7c259d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:37.959290 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539dda8-8a7e-4ac2-a56a-3db2d7c259d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:37.959944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539dda8-8a7e-4ac2-a56a-3db2d7c259d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:37.961320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539dda8-8a7e-4ac2-a56a-3db2d7c259d9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-c539dda8-8a7e-4ac2-a56a-3db2d7c259d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:37.973672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:37.974777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:37.975655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 487/960] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:37.981176 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465ca08-78c4-4815-83ca-c9023f5ab414 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:38.082527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465ca08-78c4-4815-83ca-c9023f5ab414 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:38.117195 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e2b2d-f28d-498a-9c90-709a19b35c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:38.117870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e2b2d-f28d-498a-9c90-709a19b35c47 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:38.118188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e2b2d-f28d-498a-9c90-709a19b35c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:38.118469 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e2b2d-f28d-498a-9c90-709a19b35c47 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:38.119197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e2b2d-f28d-498a-9c90-709a19b35c47 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:38.119509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e2b2d-f28d-498a-9c90-709a19b35c47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:38.121124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e2b2d-f28d-498a-9c90-709a19b35c47 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:38.126597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 474/961] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jun 20 10:52:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:38.131963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539dda8-8a7e-4ac2-a56a-3db2d7c259d9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:38.198181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:38.234417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539dda8-8a7e-4ac2-a56a-3db2d7c259d9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:38.268453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366766d-f52d-48f1-9ef1-1df2302ba07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:38.269138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366766d-f52d-48f1-9ef1-1df2302ba07b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:38.269450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366766d-f52d-48f1-9ef1-1df2302ba07b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:38.269754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366766d-f52d-48f1-9ef1-1df2302ba07b None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:38.270520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366766d-f52d-48f1-9ef1-1df2302ba07b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:38.270847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366766d-f52d-48f1-9ef1-1df2302ba07b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:38.272208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366766d-f52d-48f1-9ef1-1df2302ba07b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:38.277788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 488/962] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jun 20 10:52:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:38.283413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e2b2d-f28d-498a-9c90-709a19b35c47 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:38.319063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:38.392915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e2b2d-f28d-498a-9c90-709a19b35c47 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:38.428496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f649222-8b8f-4526-b21b-80ce2c614dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:38.429166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f649222-8b8f-4526-b21b-80ce2c614dea None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:38.429450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f649222-8b8f-4526-b21b-80ce2c614dea None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:38.429727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f649222-8b8f-4526-b21b-80ce2c614dea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:38.430525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f649222-8b8f-4526-b21b-80ce2c614dea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:38.431918 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f649222-8b8f-4526-b21b-80ce2c614dea None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:38.455239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 475/963] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jun 20 10:52:38 2022] POST /identity/v3/endpoints => generated 349 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:38.484526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8d0e4-f940-4a03-8d3e-4f9aec80858a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:38.484797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8d0e4-f940-4a03-8d3e-4f9aec80858a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:38.485058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b8d0e4-f940-4a03-8d3e-4f9aec80858a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:38.486737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 489/964] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:38.500185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea456a1b-ee8d-48d4-b9fb-27ec654e41a3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:38.500810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea456a1b-ee8d-48d4-b9fb-27ec654e41a3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:38.501239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea456a1b-ee8d-48d4-b9fb-27ec654e41a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:38.521915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ea456a1b-ee8d-48d4-b9fb-27ec654e41a3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:38.523231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ea456a1b-ee8d-48d4-b9fb-27ec654e41a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:38.605483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 476/965] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:38.632388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8d0e4-f940-4a03-8d3e-4f9aec80858a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:38.738305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366022ed-fa8e-42bc-8d36-78adb01fbe87 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:38.738610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366022ed-fa8e-42bc-8d36-78adb01fbe87 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:38.738786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366022ed-fa8e-42bc-8d36-78adb01fbe87 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:38.741175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 477/966] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:38.746449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e8a2-828e-41cc-8c92-b324473eec42 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:38.746734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e8a2-828e-41cc-8c92-b324473eec42 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:38.747010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e8a2-828e-41cc-8c92-b324473eec42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:38.749439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b8d0e4-f940-4a03-8d3e-4f9aec80858a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:38.766826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-27c2e8a2-828e-41cc-8c92-b324473eec42 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:38.768502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-27c2e8a2-828e-41cc-8c92-b324473eec42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:38.780082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:38.782690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:38.787827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892f926-c12b-4961-94e0-2b0f4d0af7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:38.788748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892f926-c12b-4961-94e0-2b0f4d0af7ce None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:38.789243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892f926-c12b-4961-94e0-2b0f4d0af7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:38.789745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892f926-c12b-4961-94e0-2b0f4d0af7ce None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:38.791472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892f926-c12b-4961-94e0-2b0f4d0af7ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:38.793373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892f926-c12b-4961-94e0-2b0f4d0af7ce None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:38.809462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.sql.core [None req-1892f926-c12b-4961-94e0-2b0f4d0af7ce None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 20 10:52:38.809462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 20 10:52:38.809462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [parameters: {'user_id': '3e6ce34315b7419d90895d93f28c2063', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 20 10:52:38.809462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80133) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1892f926-c12b-4961-94e0-2b0f4d0af7ce None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application conn.query(q) Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result.read() Jun 20 10:52:38.819454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 10:52:38.820405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application next(self.gen) Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application yield resource Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application next(self.gen) Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.session.rollback() Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.force_reraise() Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise self.value Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application session.commit() Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 20 10:52:38.821343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._prepare_impl() Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.session.flush() Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._flush(objects) Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application compat.raise_( Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application flush_context.execute() Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rec.execute(self) Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 20 10:52:38.822263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 10:52:38.823601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application conn.query(q) Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result.read() Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 10:52:38.825844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [parameters: {'user_id': '3e6ce34315b7419d90895d93f28c2063', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:38.828756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 20 10:52:38.830902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 20 10:52:38.833686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 20 10:52:38.833686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 20 10:52:38.833686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 20 10:52:38.833686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 20 10:52:38.833686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 20 10:52:38.833686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:38.833686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 490/967] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Jun 20 10:52:38 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:38.833686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892f926-c12b-4961-94e0-2b0f4d0af7ce None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:38.861814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 478/968] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:38.887157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e8a2-828e-41cc-8c92-b324473eec42 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:38.966071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892f926-c12b-4961-94e0-2b0f4d0af7ce None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:39.002847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a684d7-d5b1-404d-9c92-296f551804cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:39.003573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a684d7-d5b1-404d-9c92-296f551804cb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.003896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a684d7-d5b1-404d-9c92-296f551804cb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.004205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a684d7-d5b1-404d-9c92-296f551804cb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.012847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a684d7-d5b1-404d-9c92-296f551804cb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:39.014695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a684d7-d5b1-404d-9c92-296f551804cb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:39.024605 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e8a2-828e-41cc-8c92-b324473eec42 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-c9a684d7-d5b1-404d-9c92-296f551804cb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:39.029629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:39.030632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:39.031561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 491/969] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:39.036940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a684d7-d5b1-404d-9c92-296f551804cb None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:39.068791 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc6f15-69ad-403b-878a-a39c511ff647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:39.069588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc6f15-69ad-403b-878a-a39c511ff647 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.069864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc6f15-69ad-403b-878a-a39c511ff647 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.070157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc6f15-69ad-403b-878a-a39c511ff647 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.070819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc6f15-69ad-403b-878a-a39c511ff647 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:39.072316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc6f15-69ad-403b-878a-a39c511ff647 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-1dfc6f15-69ad-403b-878a-a39c511ff647 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:39.085142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:39.086201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:39.087090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 479/970] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:39.093584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc6f15-69ad-403b-878a-a39c511ff647 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:39.104734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:39.111100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:39.159340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a684d7-d5b1-404d-9c92-296f551804cb None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:39.195756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b19ea-a00b-434d-a21d-d36c6120fb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:39.196549 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b19ea-a00b-434d-a21d-d36c6120fb53 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.196869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b19ea-a00b-434d-a21d-d36c6120fb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.197173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b19ea-a00b-434d-a21d-d36c6120fb53 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.198228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b19ea-a00b-434d-a21d-d36c6120fb53 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:39.198618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b19ea-a00b-434d-a21d-d36c6120fb53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:39.200110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b19ea-a00b-434d-a21d-d36c6120fb53 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:39.219435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc6f15-69ad-403b-878a-a39c511ff647 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:39.249461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 492/971] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Jun 20 10:52:39 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:39.256631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0151fc-803a-4731-a763-e6c2eb196e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:39.257709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0151fc-803a-4731-a763-e6c2eb196e10 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.258230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0151fc-803a-4731-a763-e6c2eb196e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.258712 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0151fc-803a-4731-a763-e6c2eb196e10 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.259662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0151fc-803a-4731-a763-e6c2eb196e10 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:39.260238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0151fc-803a-4731-a763-e6c2eb196e10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:39.261980 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0151fc-803a-4731-a763-e6c2eb196e10 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:39.268617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 480/972] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:39.274884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b19ea-a00b-434d-a21d-d36c6120fb53 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:39.312108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72acf3-1e64-4d7a-ba16-1b57f9863551 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.312471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72acf3-1e64-4d7a-ba16-1b57f9863551 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.312668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72acf3-1e64-4d7a-ba16-1b57f9863551 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.314273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 481/973] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:52:39.320010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa22fbf-76d0-4cd3-95ad-61eff378f5bb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.320648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa22fbf-76d0-4cd3-95ad-61eff378f5bb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.321139 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa22fbf-76d0-4cd3-95ad-61eff378f5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.340029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5aa22fbf-76d0-4cd3-95ad-61eff378f5bb None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:39.341859 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5aa22fbf-76d0-4cd3-95ad-61eff378f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:39.394081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b19ea-a00b-434d-a21d-d36c6120fb53 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:39.406970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:39.422135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 482/974] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:39.438356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7661b-bada-4110-92e8-b5373b4856f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:39.439033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7661b-bada-4110-92e8-b5373b4856f9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.439315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7661b-bada-4110-92e8-b5373b4856f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.439587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7661b-bada-4110-92e8-b5373b4856f9 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.440297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7661b-bada-4110-92e8-b5373b4856f9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:39.440603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7661b-bada-4110-92e8-b5373b4856f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:39.441975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7661b-bada-4110-92e8-b5373b4856f9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:39.445670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa22fbf-76d0-4cd3-95ad-61eff378f5bb None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:39.449013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:39.451185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 493/975] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:39.458268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7661b-bada-4110-92e8-b5373b4856f9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:39.580298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7661b-bada-4110-92e8-b5373b4856f9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:39.585744 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa22fbf-76d0-4cd3-95ad-61eff378f5bb None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:39.623490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3da9f8-57ef-497b-bcac-87a3e5b71b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:39.623723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dec8b2-c253-4ea6-89ff-ec0476066618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:39.624319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dec8b2-c253-4ea6-89ff-ec0476066618 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.624423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3da9f8-57ef-497b-bcac-87a3e5b71b38 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.624597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dec8b2-c253-4ea6-89ff-ec0476066618 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.624694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3da9f8-57ef-497b-bcac-87a3e5b71b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.624868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dec8b2-c253-4ea6-89ff-ec0476066618 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.624962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3da9f8-57ef-497b-bcac-87a3e5b71b38 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.625669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dec8b2-c253-4ea6-89ff-ec0476066618 None admin] RBAC: Adding query filter params (interface=public, service_id=75d663fd12f246c9bf004fa489bec888, region_id=RegionOne) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:39.625982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dec8b2-c253-4ea6-89ff-ec0476066618 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:39.627789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dec8b2-c253-4ea6-89ff-ec0476066618 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:39.639777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 494/976] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jun 20 10:52:39 2022] GET /identity/v3/endpoints?service_id=75d663fd12f246c9bf004fa489bec888&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:39.639978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3da9f8-57ef-497b-bcac-87a3e5b71b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:39.640866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dec8b2-c253-4ea6-89ff-ec0476066618 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-4f3da9f8-57ef-497b-bcac-87a3e5b71b38 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:39.645420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:39.646579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:39.647420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 483/977] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jun 20 10:52:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:39.652438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3da9f8-57ef-497b-bcac-87a3e5b71b38 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:39.694940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:39.719894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:39.758231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dec8b2-c253-4ea6-89ff-ec0476066618 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:39.766631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3da9f8-57ef-497b-bcac-87a3e5b71b38 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:39.791561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858e1a7-f025-4fca-a516-7b8dc96de1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:39.792255 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858e1a7-f025-4fca-a516-7b8dc96de1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.792537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858e1a7-f025-4fca-a516-7b8dc96de1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.792841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858e1a7-f025-4fca-a516-7b8dc96de1b9 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.793527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858e1a7-f025-4fca-a516-7b8dc96de1b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:39.794941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858e1a7-f025-4fca-a516-7b8dc96de1b9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:39.802189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 495/978] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jun 20 10:52:39 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 10:52:39.804021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345805b8-fcd7-46ae-8b81-c2388568cbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:39.804893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345805b8-fcd7-46ae-8b81-c2388568cbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.805348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345805b8-fcd7-46ae-8b81-c2388568cbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.805805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345805b8-fcd7-46ae-8b81-c2388568cbc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.806785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345805b8-fcd7-46ae-8b81-c2388568cbc4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:39.807952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345805b8-fcd7-46ae-8b81-c2388568cbc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:39.809632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345805b8-fcd7-46ae-8b81-c2388568cbc4 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:39.819839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 484/979] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jun 20 10:52:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:39.825472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858e1a7-f025-4fca-a516-7b8dc96de1b9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:39.942916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858e1a7-f025-4fca-a516-7b8dc96de1b9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:39.944674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:39.981586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e3632-4795-45bf-ab83-038a96f99fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:39.982428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e3632-4795-45bf-ab83-038a96f99fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:39.982702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e3632-4795-45bf-ab83-038a96f99fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:39.982969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e3632-4795-45bf-ab83-038a96f99fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:39.994122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e3632-4795-45bf-ab83-038a96f99fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:39.995958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e3632-4795-45bf-ab83-038a96f99fd4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-557e3632-4795-45bf-ab83-038a96f99fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:40.013654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:40.014626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:40.015926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:40.015926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:40.015926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:40.015926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:40.015926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 496/980] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:40.020758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345805b8-fcd7-46ae-8b81-c2388568cbc4 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:40.125293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345805b8-fcd7-46ae-8b81-c2388568cbc4 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:40.155076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:40.164948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089dfec-53d3-4ae1-b608-ce12ab9b1fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:40.165543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089dfec-53d3-4ae1-b608-ce12ab9b1fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:40.165818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089dfec-53d3-4ae1-b608-ce12ab9b1fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:40.166113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089dfec-53d3-4ae1-b608-ce12ab9b1fa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:40.166948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089dfec-53d3-4ae1-b608-ce12ab9b1fa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:40.167254 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089dfec-53d3-4ae1-b608-ce12ab9b1fa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:40.168707 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089dfec-53d3-4ae1-b608-ce12ab9b1fa5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:40.186187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 485/981] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:40.187454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e3632-4795-45bf-ab83-038a96f99fd4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:40.299037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e3632-4795-45bf-ab83-038a96f99fd4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:40.335495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ab700-c0d8-4b4c-a7a4-6e9d6ba24386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:40.336329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ab700-c0d8-4b4c-a7a4-6e9d6ba24386 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:40.336600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ab700-c0d8-4b4c-a7a4-6e9d6ba24386 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:40.336859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ab700-c0d8-4b4c-a7a4-6e9d6ba24386 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:40.344678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ab700-c0d8-4b4c-a7a4-6e9d6ba24386 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:40.346130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ab700-c0d8-4b4c-a7a4-6e9d6ba24386 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-ea4ab700-c0d8-4b4c-a7a4-6e9d6ba24386 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:40.361382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:40.362616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:40.363574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:40.363574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:40.363574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:40.363574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:40.363574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:40.363574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:40.363574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:40.363574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:40.363574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 10:52:40.363574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:40.364602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 497/982] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jun 20 10:52:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:40.368345 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089dfec-53d3-4ae1-b608-ce12ab9b1fa5 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:40.470027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089dfec-53d3-4ae1-b608-ce12ab9b1fa5 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:40.505377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b107476-5504-42eb-af4b-416362442960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:40.506033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b107476-5504-42eb-af4b-416362442960 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:40.506357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b107476-5504-42eb-af4b-416362442960 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:40.506639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b107476-5504-42eb-af4b-416362442960 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:40.507467 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b107476-5504-42eb-af4b-416362442960 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:40.507774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b107476-5504-42eb-af4b-416362442960 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:40.509239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b107476-5504-42eb-af4b-416362442960 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:40.527130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 486/983] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Jun 20 10:52:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:40.532715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ab700-c0d8-4b4c-a7a4-6e9d6ba24386 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:40.548189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187cafa-1d7e-4f06-9264-ae43ba6698ec None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:40.548398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187cafa-1d7e-4f06-9264-ae43ba6698ec None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:40.548693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187cafa-1d7e-4f06-9264-ae43ba6698ec None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:40.549740 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:40.550122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 487/984] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:52:40.555753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8611f90-3517-452e-b076-a2481c70e554 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:40.556041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8611f90-3517-452e-b076-a2481c70e554 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:40.556315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8611f90-3517-452e-b076-a2481c70e554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:40.573759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e8611f90-3517-452e-b076-a2481c70e554 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:40.574970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e8611f90-3517-452e-b076-a2481c70e554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:40.615714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:40.641159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ab700-c0d8-4b4c-a7a4-6e9d6ba24386 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:40.643056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 488/985] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:40.665428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8611f90-3517-452e-b076-a2481c70e554 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:40.679589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc805-19ac-4345-baa7-717a901e289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:40.680440 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc805-19ac-4345-baa7-717a901e289f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:40.680915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc805-19ac-4345-baa7-717a901e289f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:40.681374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc805-19ac-4345-baa7-717a901e289f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:40.761790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedc805-19ac-4345-baa7-717a901e289f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:40.763333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedc805-19ac-4345-baa7-717a901e289f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:40.767206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8611f90-3517-452e-b076-a2481c70e554 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:40.779216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:40.804302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f931ef-818a-4058-964c-840d172644ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:40.805314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f931ef-818a-4058-964c-840d172644ee None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:40.805426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f931ef-818a-4058-964c-840d172644ee None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:40.805649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f931ef-818a-4058-964c-840d172644ee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:40.806324 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f931ef-818a-4058-964c-840d172644ee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:40.807857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f931ef-818a-4058-964c-840d172644ee None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-30f931ef-818a-4058-964c-840d172644ee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:40.819718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:40.820721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:40.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 489/986] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:40.827079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f931ef-818a-4058-964c-840d172644ee None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:40.870384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:40.877018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 498/987] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:40 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:40.882524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc805-19ac-4345-baa7-717a901e289f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:40.938603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f931ef-818a-4058-964c-840d172644ee None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:40.980967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21f48c-47c7-4960-85b0-f5ba01902be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:40.981655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21f48c-47c7-4960-85b0-f5ba01902be0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:40.981932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21f48c-47c7-4960-85b0-f5ba01902be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:40.982238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21f48c-47c7-4960-85b0-f5ba01902be0 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:40.982956 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21f48c-47c7-4960-85b0-f5ba01902be0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:40.983262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21f48c-47c7-4960-85b0-f5ba01902be0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:40.984648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21f48c-47c7-4960-85b0-f5ba01902be0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:40.994241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 490/988] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jun 20 10:52:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:40.996026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21f48c-47c7-4960-85b0-f5ba01902be0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:40.998841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc805-19ac-4345-baa7-717a901e289f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:41.015749 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:41.037156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd80e3-2667-487d-9aab-1c6294c2bc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:41.037962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd80e3-2667-487d-9aab-1c6294c2bc26 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:41.038273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd80e3-2667-487d-9aab-1c6294c2bc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:41.038709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd80e3-2667-487d-9aab-1c6294c2bc26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:41.050318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd80e3-2667-487d-9aab-1c6294c2bc26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:41.051812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd80e3-2667-487d-9aab-1c6294c2bc26 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1ddd80e3-2667-487d-9aab-1c6294c2bc26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:41.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:41.070612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:41.071494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:41.071494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:41.071494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:41.071494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:41.071494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 499/989] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:41.076921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd80e3-2667-487d-9aab-1c6294c2bc26 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:41.112227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21f48c-47c7-4960-85b0-f5ba01902be0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:41.115214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:41.148302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e3526-b9aa-46ff-a1ae-4c66add02d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:41.149305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e3526-b9aa-46ff-a1ae-4c66add02d44 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:41.149305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e3526-b9aa-46ff-a1ae-4c66add02d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:41.149568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e3526-b9aa-46ff-a1ae-4c66add02d44 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:41.150254 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e3526-b9aa-46ff-a1ae-4c66add02d44 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:41.150568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e3526-b9aa-46ff-a1ae-4c66add02d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:41.152145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e3526-b9aa-46ff-a1ae-4c66add02d44 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:41.161804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 491/990] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jun 20 10:52:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:41.163057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e3526-b9aa-46ff-a1ae-4c66add02d44 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:41.188570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd80e3-2667-487d-9aab-1c6294c2bc26 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:41.226581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d7862-4351-484a-a9ba-0851dc0597b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:41.227607 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d7862-4351-484a-a9ba-0851dc0597b8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:41.228063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d7862-4351-484a-a9ba-0851dc0597b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:41.228347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d7862-4351-484a-a9ba-0851dc0597b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:41.229186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d7862-4351-484a-a9ba-0851dc0597b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:41.229657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d7862-4351-484a-a9ba-0851dc0597b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:41.230891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951d7862-4351-484a-a9ba-0851dc0597b8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:41.249606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 500/991] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:41.250990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d7862-4351-484a-a9ba-0851dc0597b8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:41.275417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e3526-b9aa-46ff-a1ae-4c66add02d44 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:41.282387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:41.313035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f087b-34cd-40c6-9ce0-e8ef514f44ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:41.313709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f087b-34cd-40c6-9ce0-e8ef514f44ab None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:41.314002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f087b-34cd-40c6-9ce0-e8ef514f44ab None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:41.314320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f087b-34cd-40c6-9ce0-e8ef514f44ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:41.315134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f087b-34cd-40c6-9ce0-e8ef514f44ab None admin] RBAC: Adding query filter params (interface=admin, service_id=388f3980d2304cb9ac4022e3282da635, region_id=RegionOne) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:41.315448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f087b-34cd-40c6-9ce0-e8ef514f44ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:41.316830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f087b-34cd-40c6-9ce0-e8ef514f44ab None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:41.328292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 492/992] 10.222.0.33 () {56 vars in 1249 bytes} [Mon Jun 20 10:52:41 2022] GET /identity/v3/endpoints?service_id=388f3980d2304cb9ac4022e3282da635&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:41.329333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f087b-34cd-40c6-9ce0-e8ef514f44ab None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:41.364592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:41.367997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d7862-4351-484a-a9ba-0851dc0597b8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:41.403863 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb341af9-bd78-4533-86cb-1d8389d5570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:41.404672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb341af9-bd78-4533-86cb-1d8389d5570f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:41.404957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb341af9-bd78-4533-86cb-1d8389d5570f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:41.405230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb341af9-bd78-4533-86cb-1d8389d5570f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:41.412389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb341af9-bd78-4533-86cb-1d8389d5570f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:41.413847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb341af9-bd78-4533-86cb-1d8389d5570f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-bb341af9-bd78-4533-86cb-1d8389d5570f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:41.428153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:41.429095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:41.429991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:41.429991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:41.429991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:41.429991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:41.429991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:41.429991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:41.429991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:41.429991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:41.429991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 501/993] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:41.434959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb341af9-bd78-4533-86cb-1d8389d5570f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:41.445016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f087b-34cd-40c6-9ce0-e8ef514f44ab None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:41.480436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338658d-e7e7-4d1d-b999-368c3d953acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:41.481372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338658d-e7e7-4d1d-b999-368c3d953acb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:41.481647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338658d-e7e7-4d1d-b999-368c3d953acb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:41.481929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338658d-e7e7-4d1d-b999-368c3d953acb None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:41.482649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338658d-e7e7-4d1d-b999-368c3d953acb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:41.484047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338658d-e7e7-4d1d-b999-368c3d953acb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:41.498744 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 493/994] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jun 20 10:52:41 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 10:52:41.531254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:41.549341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb341af9-bd78-4533-86cb-1d8389d5570f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:41.587864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacebea8-af39-42a3-8224-9f0837a7bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:41.588792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacebea8-af39-42a3-8224-9f0837a7bcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:41.589075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacebea8-af39-42a3-8224-9f0837a7bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:41.589354 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacebea8-af39-42a3-8224-9f0837a7bcc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:41.590158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacebea8-af39-42a3-8224-9f0837a7bcc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:41.590484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacebea8-af39-42a3-8224-9f0837a7bcc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:41.591955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacebea8-af39-42a3-8224-9f0837a7bcc4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:41.605563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 502/995] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:41.611656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338658d-e7e7-4d1d-b999-368c3d953acb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:41.720189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:41.728453 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338658d-e7e7-4d1d-b999-368c3d953acb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:41.763673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa00fba-cf52-4725-a3e0-acc6531f50f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:41.764342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa00fba-cf52-4725-a3e0-acc6531f50f9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:41.764643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa00fba-cf52-4725-a3e0-acc6531f50f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:41.764921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa00fba-cf52-4725-a3e0-acc6531f50f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:41.765754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa00fba-cf52-4725-a3e0-acc6531f50f9 None admin] RBAC: Adding query filter params (role.id=6db9b3d9148e4e37bded8945be0f6f6f, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=aca3b45e80764c3b8254e24d09ff0d4c) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:41.766079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa00fba-cf52-4725-a3e0-acc6531f50f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:41.767562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa00fba-cf52-4725-a3e0-acc6531f50f9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:41.773899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 494/996] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:41 2022] GET /identity/v3/role_assignments?user.id=aca3b45e80764c3b8254e24d09ff0d4c&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=6db9b3d9148e4e37bded8945be0f6f6f => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:41.866873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc580d69-c6c9-42ab-b477-5dd05856947f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:41.867086 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc580d69-c6c9-42ab-b477-5dd05856947f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:41.867310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc580d69-c6c9-42ab-b477-5dd05856947f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:41.868925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 503/997] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:41.874763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7bdf4-9a11-4802-802f-3be1b753c2a4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:41.875114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7bdf4-9a11-4802-802f-3be1b753c2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:41.875292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7bdf4-9a11-4802-802f-3be1b753c2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:41.893518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-25d7bdf4-9a11-4802-802f-3be1b753c2a4 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:41.894764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-25d7bdf4-9a11-4802-802f-3be1b753c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:41.981199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 495/998] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:42.004017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc580d69-c6c9-42ab-b477-5dd05856947f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:42.110531 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc580d69-c6c9-42ab-b477-5dd05856947f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:42.145075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f6144-a02e-4614-a355-38110bd8b378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:42.145806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f6144-a02e-4614-a355-38110bd8b378 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:42.146108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f6144-a02e-4614-a355-38110bd8b378 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:42.146377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f6144-a02e-4614-a355-38110bd8b378 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:42.147047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f6144-a02e-4614-a355-38110bd8b378 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:42.148426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f6144-a02e-4614-a355-38110bd8b378 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-613f6144-a02e-4614-a355-38110bd8b378 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:42.161395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:42.162673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:42.163510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 504/999] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:42.170556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7bdf4-9a11-4802-802f-3be1b753c2a4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:42.260976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:42.274748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7bdf4-9a11-4802-802f-3be1b753c2a4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:42.309398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1abf9-950b-4a90-b859-6c498cd116a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:42.310077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1abf9-950b-4a90-b859-6c498cd116a0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:42.310386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1abf9-950b-4a90-b859-6c498cd116a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:42.310661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1abf9-950b-4a90-b859-6c498cd116a0 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:42.311375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1abf9-950b-4a90-b859-6c498cd116a0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:42.311683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1abf9-950b-4a90-b859-6c498cd116a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:42.313074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1abf9-950b-4a90-b859-6c498cd116a0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:42.318068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 496/1000] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:42.323743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f6144-a02e-4614-a355-38110bd8b378 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:42.337256 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:42.430276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f6144-a02e-4614-a355-38110bd8b378 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:42.467854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818c242-450b-42aa-a20c-7fb628cb0821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:42.468527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818c242-450b-42aa-a20c-7fb628cb0821 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:42.468809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818c242-450b-42aa-a20c-7fb628cb0821 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:42.469085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818c242-450b-42aa-a20c-7fb628cb0821 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:42.469774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818c242-450b-42aa-a20c-7fb628cb0821 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:42.470076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818c242-450b-42aa-a20c-7fb628cb0821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:42.471478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818c242-450b-42aa-a20c-7fb628cb0821 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:42.476953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 505/1001] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Jun 20 10:52:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:42.483850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1abf9-950b-4a90-b859-6c498cd116a0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:42.588210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1abf9-950b-4a90-b859-6c498cd116a0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:42.626482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba1e9b-108b-422e-bbbf-6cccb0226016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:42.627149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1e9b-108b-422e-bbbf-6cccb0226016 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:42.627427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1e9b-108b-422e-bbbf-6cccb0226016 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:42.627700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba1e9b-108b-422e-bbbf-6cccb0226016 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:42.628456 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba1e9b-108b-422e-bbbf-6cccb0226016 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:42.629831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba1e9b-108b-422e-bbbf-6cccb0226016 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:42.650522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 497/1002] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jun 20 10:52:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:43.564927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb858b-cdbf-4c4a-989d-d2019861c24d None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:43.565478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb858b-cdbf-4c4a-989d-d2019861c24d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:43.565478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb858b-cdbf-4c4a-989d-d2019861c24d None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:43.567084 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 506/1003] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:43.572159 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019d564-ced4-4dfa-a899-d76b2df4a201 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:43.572729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019d564-ced4-4dfa-a899-d76b2df4a201 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:43.572829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019d564-ced4-4dfa-a899-d76b2df4a201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:43.591229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6019d564-ced4-4dfa-a899-d76b2df4a201 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:43.592406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6019d564-ced4-4dfa-a899-d76b2df4a201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:43.594413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:43.673985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 498/1004] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:43 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:43.697311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb858b-cdbf-4c4a-989d-d2019861c24d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:43.779713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:43.807793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb858b-cdbf-4c4a-989d-d2019861c24d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:43.808758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f501eb-0f29-4a3a-82c3-f8f1d9b85df7 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:43.809037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f501eb-0f29-4a3a-82c3-f8f1d9b85df7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:43.809316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f501eb-0f29-4a3a-82c3-f8f1d9b85df7 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:43.810989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 499/1005] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:52:43.815951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380957b1-2f73-4228-8ec9-d0fbaf082442 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:43.816167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380957b1-2f73-4228-8ec9-d0fbaf082442 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:43.816451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380957b1-2f73-4228-8ec9-d0fbaf082442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:43.833916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-380957b1-2f73-4228-8ec9-d0fbaf082442 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:43.835187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-380957b1-2f73-4228-8ec9-d0fbaf082442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:43.846832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536bdf9-e0ce-4ff6-ae82-e44b9ee088cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:43.847980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536bdf9-e0ce-4ff6-ae82-e44b9ee088cd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:43.848430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536bdf9-e0ce-4ff6-ae82-e44b9ee088cd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:43.848696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536bdf9-e0ce-4ff6-ae82-e44b9ee088cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:43.849362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536bdf9-e0ce-4ff6-ae82-e44b9ee088cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:43.850761 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536bdf9-e0ce-4ff6-ae82-e44b9ee088cd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-0536bdf9-e0ce-4ff6-ae82-e44b9ee088cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:43.863356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:43.864390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:43.865380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 507/1006] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Jun 20 10:52:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:43.870522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536bdf9-e0ce-4ff6-ae82-e44b9ee088cd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:43.918516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 500/1007] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:43 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:43.935421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380957b1-2f73-4228-8ec9-d0fbaf082442 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:43.991773 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536bdf9-e0ce-4ff6-ae82-e44b9ee088cd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:43.999632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:44.026380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4d699-ff92-4b42-be50-cff1b5b02239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:44.027058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4d699-ff92-4b42-be50-cff1b5b02239 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:44.027338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4d699-ff92-4b42-be50-cff1b5b02239 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:44.027614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4d699-ff92-4b42-be50-cff1b5b02239 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:44.028319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4d699-ff92-4b42-be50-cff1b5b02239 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:44.028625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4d699-ff92-4b42-be50-cff1b5b02239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:44.029997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4d699-ff92-4b42-be50-cff1b5b02239 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:44.039860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 508/1008] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jun 20 10:52:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:44.040824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4d699-ff92-4b42-be50-cff1b5b02239 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:44.043750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:44.048548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380957b1-2f73-4228-8ec9-d0fbaf082442 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:44.084118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93556d1b-68d1-4fc3-b8f0-c87c5f415837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:44.084848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93556d1b-68d1-4fc3-b8f0-c87c5f415837 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:44.085130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93556d1b-68d1-4fc3-b8f0-c87c5f415837 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:44.085410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93556d1b-68d1-4fc3-b8f0-c87c5f415837 None admin] PATH_INFO: `/v3/services/image` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:44.086069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93556d1b-68d1-4fc3-b8f0-c87c5f415837 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:44.087475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93556d1b-68d1-4fc3-b8f0-c87c5f415837 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-93556d1b-68d1-4fc3-b8f0-c87c5f415837 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:44.099101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:44.100077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 10:52:44.100964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:44.105524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 501/1009] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Jun 20 10:52:43 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:44.106383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93556d1b-68d1-4fc3-b8f0-c87c5f415837 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:44.153073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4d699-ff92-4b42-be50-cff1b5b02239 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:44.187187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41c622-6261-4b40-a03e-14cdc5f1fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:44.188217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c622-6261-4b40-a03e-14cdc5f1fa45 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:44.188700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c622-6261-4b40-a03e-14cdc5f1fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:44.189196 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41c622-6261-4b40-a03e-14cdc5f1fa45 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:44.190228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41c622-6261-4b40-a03e-14cdc5f1fa45 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:44.190891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41c622-6261-4b40-a03e-14cdc5f1fa45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:44.192341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41c622-6261-4b40-a03e-14cdc5f1fa45 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:44.202711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 509/1010] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Jun 20 10:52:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:44.203939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41c622-6261-4b40-a03e-14cdc5f1fa45 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:44.221269 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93556d1b-68d1-4fc3-b8f0-c87c5f415837 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:44.247201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:44.257649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d1f5f-eda0-42b5-9719-f8b7af7d8a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:44.258751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1f5f-eda0-42b5-9719-f8b7af7d8a61 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:44.259487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1f5f-eda0-42b5-9719-f8b7af7d8a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:44.259953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1f5f-eda0-42b5-9719-f8b7af7d8a61 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:44.261018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d1f5f-eda0-42b5-9719-f8b7af7d8a61 None admin] RBAC: Adding query filter params (name=image) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:44.261519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d1f5f-eda0-42b5-9719-f8b7af7d8a61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:44.263801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d1f5f-eda0-42b5-9719-f8b7af7d8a61 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:44.274257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 502/1011] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:44.276036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d1f5f-eda0-42b5-9719-f8b7af7d8a61 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:44.278985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:44.316032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41c622-6261-4b40-a03e-14cdc5f1fa45 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:44.350583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b132812-8b90-4c6f-b934-e6150548390d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:44.351643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b132812-8b90-4c6f-b934-e6150548390d None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:44.351932 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b132812-8b90-4c6f-b934-e6150548390d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:44.352202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b132812-8b90-4c6f-b934-e6150548390d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:44.353084 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b132812-8b90-4c6f-b934-e6150548390d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:44.354514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b132812-8b90-4c6f-b934-e6150548390d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:44.382127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 510/1012] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jun 20 10:52:44 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 10:52:44.398665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d1f5f-eda0-42b5-9719-f8b7af7d8a61 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:44.445405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35ee53-e44f-4173-aa49-f46ba276a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:44.446109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35ee53-e44f-4173-aa49-f46ba276a60f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:44.446404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35ee53-e44f-4173-aa49-f46ba276a60f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:44.446679 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35ee53-e44f-4173-aa49-f46ba276a60f None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:44.447406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35ee53-e44f-4173-aa49-f46ba276a60f None admin] RBAC: Adding query filter params (type=image) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:44.447909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35ee53-e44f-4173-aa49-f46ba276a60f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:44.449377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35ee53-e44f-4173-aa49-f46ba276a60f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:44.455012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 503/1013] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:44 2022] GET /identity/v3/services?type=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:44.791260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be9e9b-50b0-4fe0-9b3a-68a010da25ff None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:44.792034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be9e9b-50b0-4fe0-9b3a-68a010da25ff None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:44.792122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be9e9b-50b0-4fe0-9b3a-68a010da25ff None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:44.793513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 511/1014] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:44.799055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58b376-9b22-4f3d-8d71-dc7735fea341 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:44.799472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58b376-9b22-4f3d-8d71-dc7735fea341 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:44.799624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58b376-9b22-4f3d-8d71-dc7735fea341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:44.817488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ad58b376-9b22-4f3d-8d71-dc7735fea341 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:44.818713 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ad58b376-9b22-4f3d-8d71-dc7735fea341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:44.871211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:44.897754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 504/1015] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:44 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:44.918600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be9e9b-50b0-4fe0-9b3a-68a010da25ff None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:45.029581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be9e9b-50b0-4fe0-9b3a-68a010da25ff None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:45.056702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:45.067913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92434a-5742-4e5b-b5db-44fd6ab1244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:45.068762 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92434a-5742-4e5b-b5db-44fd6ab1244c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:45.069050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92434a-5742-4e5b-b5db-44fd6ab1244c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:45.069327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92434a-5742-4e5b-b5db-44fd6ab1244c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:45.081219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92434a-5742-4e5b-b5db-44fd6ab1244c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-0d92434a-5742-4e5b-b5db-44fd6ab1244c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:45.089940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:45.090907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:45.091807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:45.094481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 512/1016] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jun 20 10:52:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:45.097187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58b376-9b22-4f3d-8d71-dc7735fea341 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:45.203305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58b376-9b22-4f3d-8d71-dc7735fea341 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:45.239211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a7874-438d-40e0-9055-79798dcf6dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:45.239883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a7874-438d-40e0-9055-79798dcf6dda None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:45.240166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a7874-438d-40e0-9055-79798dcf6dda None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:45.240444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a7874-438d-40e0-9055-79798dcf6dda None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:45.241190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a7874-438d-40e0-9055-79798dcf6dda None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:45.241501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a7874-438d-40e0-9055-79798dcf6dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:45.242913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a7874-438d-40e0-9055-79798dcf6dda None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:45.251475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 505/1017] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jun 20 10:52:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:45.257164 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92434a-5742-4e5b-b5db-44fd6ab1244c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:45.359956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92434a-5742-4e5b-b5db-44fd6ab1244c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:45.396971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b67cb-8c58-4121-89cf-bb27816d8ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:45.397775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b67cb-8c58-4121-89cf-bb27816d8ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:45.398050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b67cb-8c58-4121-89cf-bb27816d8ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:45.398348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b67cb-8c58-4121-89cf-bb27816d8ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:45.409143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b67cb-8c58-4121-89cf-bb27816d8ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:45.410643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b67cb-8c58-4121-89cf-bb27816d8ee5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-4f5b67cb-8c58-4121-89cf-bb27816d8ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:45.428544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:45.429449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:45.430756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:45.430756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:45.430756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:45.430756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:45.434416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 513/1018] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:45.436248 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a7874-438d-40e0-9055-79798dcf6dda None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:45.485443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:45.543810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a7874-438d-40e0-9055-79798dcf6dda None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:45.582212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba512e-06ac-42d7-b56f-cfdabd2f7e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:45.582883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba512e-06ac-42d7-b56f-cfdabd2f7e65 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:45.583163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba512e-06ac-42d7-b56f-cfdabd2f7e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:45.583443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba512e-06ac-42d7-b56f-cfdabd2f7e65 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:45.584178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba512e-06ac-42d7-b56f-cfdabd2f7e65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:45.584479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba512e-06ac-42d7-b56f-cfdabd2f7e65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:45.585879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba512e-06ac-42d7-b56f-cfdabd2f7e65 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:45.602247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 506/1019] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:45.604475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b67cb-8c58-4121-89cf-bb27816d8ee5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:45.645603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:45.712351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b67cb-8c58-4121-89cf-bb27816d8ee5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:45.762290 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fc57e-933e-4f0b-a7e1-2f842f85cc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:45.763088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fc57e-933e-4f0b-a7e1-2f842f85cc7a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:45.763334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fc57e-933e-4f0b-a7e1-2f842f85cc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:45.763645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fc57e-933e-4f0b-a7e1-2f842f85cc7a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:45.771510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fc57e-933e-4f0b-a7e1-2f842f85cc7a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:45.772957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fc57e-933e-4f0b-a7e1-2f842f85cc7a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-8d6fc57e-933e-4f0b-a7e1-2f842f85cc7a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:45.786872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:45.787798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:45.788715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:45.788715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:45.788715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:45.788715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:45.788715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:45.788715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:45.788715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:45.788715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:45.788715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:45.788715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:45.790457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 514/1020] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jun 20 10:52:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:45.794184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba512e-06ac-42d7-b56f-cfdabd2f7e65 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:45.908704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba512e-06ac-42d7-b56f-cfdabd2f7e65 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:45.922275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:45.950030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557e53b-a3f4-4859-9291-9f5796c35554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:45.950764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557e53b-a3f4-4859-9291-9f5796c35554 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:45.951018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557e53b-a3f4-4859-9291-9f5796c35554 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:45.951323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557e53b-a3f4-4859-9291-9f5796c35554 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:45.952173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557e53b-a3f4-4859-9291-9f5796c35554 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:45.952484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557e53b-a3f4-4859-9291-9f5796c35554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:45.953940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557e53b-a3f4-4859-9291-9f5796c35554 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:45.973043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 507/1021] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jun 20 10:52:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:45.978712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fc57e-933e-4f0b-a7e1-2f842f85cc7a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:46.083365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:46.089396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fc57e-933e-4f0b-a7e1-2f842f85cc7a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:46.125055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bf7a9-9fd2-4528-b586-a6e4fa6e96d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:46.125718 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bf7a9-9fd2-4528-b586-a6e4fa6e96d9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.126002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bf7a9-9fd2-4528-b586-a6e4fa6e96d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.126316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bf7a9-9fd2-4528-b586-a6e4fa6e96d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.197393 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bf7a9-9fd2-4528-b586-a6e4fa6e96d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:46.199392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bf7a9-9fd2-4528-b586-a6e4fa6e96d9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:46.299375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:46.304970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 515/1022] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:45 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:46.310729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557e53b-a3f4-4859-9291-9f5796c35554 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:46.419319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557e53b-a3f4-4859-9291-9f5796c35554 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:46.446389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925121f-3090-491e-8be7-090ef7f77ef6 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.446755 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925121f-3090-491e-8be7-090ef7f77ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.447104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925121f-3090-491e-8be7-090ef7f77ef6 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.449268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 516/1023] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:46.454466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bb679-c5a7-4a06-88df-3454b886d84f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.454752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bb679-c5a7-4a06-88df-3454b886d84f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.455027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0bb679-c5a7-4a06-88df-3454b886d84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.455603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bd93d-dbba-419b-bea2-1c3f406254f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:46.456772 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bd93d-dbba-419b-bea2-1c3f406254f1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.457043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bd93d-dbba-419b-bea2-1c3f406254f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.457335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bd93d-dbba-419b-bea2-1c3f406254f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.468276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bd93d-dbba-419b-bea2-1c3f406254f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:46.469760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bd93d-dbba-419b-bea2-1c3f406254f1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:46.472495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-dc0bb679-c5a7-4a06-88df-3454b886d84f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:46.473983 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dc0bb679-c5a7-4a06-88df-3454b886d84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-be8bd93d-dbba-419b-bea2-1c3f406254f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:46.488817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:46.489733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:46.490634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:46.490634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:46.490634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:46.490634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:46.495444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 508/1024] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:46.496030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bd93d-dbba-419b-bea2-1c3f406254f1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:46.512014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:46.542858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 517/1025] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:46 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:46.544055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca34894-145d-4929-9f17-957d4b227fe1 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.544347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca34894-145d-4929-9f17-957d4b227fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.544624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca34894-145d-4929-9f17-957d4b227fe1 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.545895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 518/1026] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:46.551143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990574eb-a187-4c0c-beb1-1dc1916cb1c4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.551424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990574eb-a187-4c0c-beb1-1dc1916cb1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.551697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990574eb-a187-4c0c-beb1-1dc1916cb1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.568783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-990574eb-a187-4c0c-beb1-1dc1916cb1c4 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:46.569960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-990574eb-a187-4c0c-beb1-1dc1916cb1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:46.600131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bd93d-dbba-419b-bea2-1c3f406254f1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:46.633667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd870e-a641-4d25-8391-29fea35eb19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:46.634371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd870e-a641-4d25-8391-29fea35eb19f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.634651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd870e-a641-4d25-8391-29fea35eb19f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.634926 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd870e-a641-4d25-8391-29fea35eb19f None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.635819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd870e-a641-4d25-8391-29fea35eb19f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:46.636129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd870e-a641-4d25-8391-29fea35eb19f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:46.637532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd870e-a641-4d25-8391-29fea35eb19f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:46.639336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 519/1027] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:46 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:46.640170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990574eb-a187-4c0c-beb1-1dc1916cb1c4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:46.648653 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 509/1028] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:46.654331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd870e-a641-4d25-8391-29fea35eb19f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:46.703336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:46.728548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:46.744920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990574eb-a187-4c0c-beb1-1dc1916cb1c4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:46.753442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd870e-a641-4d25-8391-29fea35eb19f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:46.778711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06c3d8-8c77-465c-9aca-1e0f82741eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:46.779517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06c3d8-8c77-465c-9aca-1e0f82741eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.779804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06c3d8-8c77-465c-9aca-1e0f82741eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.780080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06c3d8-8c77-465c-9aca-1e0f82741eb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.781249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06c3d8-8c77-465c-9aca-1e0f82741eb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:46.782742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06c3d8-8c77-465c-9aca-1e0f82741eb4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:46.784525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 520/1029] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:46.785117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06c3d8-8c77-465c-9aca-1e0f82741eb4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:46.791633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd3826-4ce7-4f74-8c8f-d186a974eb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:46.792435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd3826-4ce7-4f74-8c8f-d186a974eb4f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.792719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd3826-4ce7-4f74-8c8f-d186a974eb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.792998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd3826-4ce7-4f74-8c8f-d186a974eb4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.800169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd3826-4ce7-4f74-8c8f-d186a974eb4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:46.801625 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd3826-4ce7-4f74-8c8f-d186a974eb4f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-44fd3826-4ce7-4f74-8c8f-d186a974eb4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:46.815885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:46.816742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:46.817990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:46.817990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:46.817990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:46.817990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:46.817990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:46.817990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:46.817990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:46.817990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:46.817990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 510/1030] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:46.817990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd3826-4ce7-4f74-8c8f-d186a974eb4f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:46.892495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06c3d8-8c77-465c-9aca-1e0f82741eb4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:46.919014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd3826-4ce7-4f74-8c8f-d186a974eb4f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:46.925165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:46.935217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e3b56-f6dd-4afc-a7f4-7a2302831848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:46.935895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e3b56-f6dd-4afc-a7f4-7a2302831848 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.936188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e3b56-f6dd-4afc-a7f4-7a2302831848 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.936464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e3b56-f6dd-4afc-a7f4-7a2302831848 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.937195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e3b56-f6dd-4afc-a7f4-7a2302831848 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:46.938631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e3b56-f6dd-4afc-a7f4-7a2302831848 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:46.952775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 521/1031] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:52:46 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:46.953636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e3b56-f6dd-4afc-a7f4-7a2302831848 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:46.958960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:46.961053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb62d8-4101-4f12-b152-62fa443d6af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:46.961748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb62d8-4101-4f12-b152-62fa443d6af8 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:46.962043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb62d8-4101-4f12-b152-62fa443d6af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:46.962626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb62d8-4101-4f12-b152-62fa443d6af8 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:46.963569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb62d8-4101-4f12-b152-62fa443d6af8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:46.965331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb62d8-4101-4f12-b152-62fa443d6af8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:46.994210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 511/1032] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:52:46 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:47.093082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e3b56-f6dd-4afc-a7f4-7a2302831848 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:47.135673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc197a-5767-4640-9321-a88b7e113af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:47.136371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc197a-5767-4640-9321-a88b7e113af8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:47.136643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc197a-5767-4640-9321-a88b7e113af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:47.136921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc197a-5767-4640-9321-a88b7e113af8 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:47.137737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc197a-5767-4640-9321-a88b7e113af8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:47.138055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc197a-5767-4640-9321-a88b7e113af8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:47.139575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc197a-5767-4640-9321-a88b7e113af8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:47.154438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 522/1033] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:47.160771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb62d8-4101-4f12-b152-62fa443d6af8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:47.263390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb62d8-4101-4f12-b152-62fa443d6af8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:47.298569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b296a93-adf4-4b03-ad60-49339b68b678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:47.299234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b296a93-adf4-4b03-ad60-49339b68b678 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:47.299517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b296a93-adf4-4b03-ad60-49339b68b678 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:47.299782 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b296a93-adf4-4b03-ad60-49339b68b678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:47.300624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b296a93-adf4-4b03-ad60-49339b68b678 None admin] RBAC: Adding query filter params (role.id=6db9b3d9148e4e37bded8945be0f6f6f, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=37b1f13aa474493d83b3a16f76949fbc) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:47.300934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b296a93-adf4-4b03-ad60-49339b68b678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:47.302419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b296a93-adf4-4b03-ad60-49339b68b678 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:47.308520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 512/1034] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:47 2022] GET /identity/v3/role_assignments?user.id=37b1f13aa474493d83b3a16f76949fbc&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=6db9b3d9148e4e37bded8945be0f6f6f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:49.012188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00963bc-8c09-4809-914f-8ab40e9796c4 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.012723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00963bc-8c09-4809-914f-8ab40e9796c4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.012723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00963bc-8c09-4809-914f-8ab40e9796c4 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.014307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 523/1035] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:49.019384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb660136-1ca5-44e5-8967-06602335aa57 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.019793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb660136-1ca5-44e5-8967-06602335aa57 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.019896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb660136-1ca5-44e5-8967-06602335aa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.037436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-eb660136-1ca5-44e5-8967-06602335aa57 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:49.038637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-eb660136-1ca5-44e5-8967-06602335aa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:49.077867 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b2280-2874-4d2b-a444-c501078a1e74 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.078264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b2280-2874-4d2b-a444-c501078a1e74 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.078452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b2280-2874-4d2b-a444-c501078a1e74 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.080034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 524/1036] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:52:49.085626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432029-489a-48a1-a1f9-f124ee551406 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.085882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432029-489a-48a1-a1f9-f124ee551406 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.086198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d432029-489a-48a1-a1f9-f124ee551406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.086430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:49.103381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3d432029-489a-48a1-a1f9-f124ee551406 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:49.104733 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3d432029-489a-48a1-a1f9-f124ee551406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:49.115639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 513/1037] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:49.139552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb660136-1ca5-44e5-8967-06602335aa57 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:49.148344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:49.181850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 525/1038] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:49.201786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d432029-489a-48a1-a1f9-f124ee551406 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:49.249038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb660136-1ca5-44e5-8967-06602335aa57 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:49.286698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183fc55-432d-4165-b527-e2c9151a05fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:49.287402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fc55-432d-4165-b527-e2c9151a05fe None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.287681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fc55-432d-4165-b527-e2c9151a05fe None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.287958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fc55-432d-4165-b527-e2c9151a05fe None admin] PATH_INFO: `/v3/services/image` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.288608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183fc55-432d-4165-b527-e2c9151a05fe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:49.290021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183fc55-432d-4165-b527-e2c9151a05fe None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-3183fc55-432d-4165-b527-e2c9151a05fe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:49.305332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:49.306669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 10:52:49.307814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:49.308633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d432029-489a-48a1-a1f9-f124ee551406 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:49.308709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 514/1039] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Jun 20 10:52:49 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:49.308709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8e1a5-94d3-47e3-81c8-b8541a42bdc0 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.308709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8e1a5-94d3-47e3-81c8-b8541a42bdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.308709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8e1a5-94d3-47e3-81c8-b8541a42bdc0 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.310295 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 515/1040] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:49.312817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8e1a5-94d3-47e3-81c8-b8541a42bdc0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:49.342588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716e044-0da9-4e54-a608-63ac55738a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:49.343397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716e044-0da9-4e54-a608-63ac55738a62 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.343680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716e044-0da9-4e54-a608-63ac55738a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.343955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716e044-0da9-4e54-a608-63ac55738a62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.345126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716e044-0da9-4e54-a608-63ac55738a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:49.346596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716e044-0da9-4e54-a608-63ac55738a62 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:49.348236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 526/1041] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:49.349435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b7d0-736c-4d11-9ea4-3a7611ecc17f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.349721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b7d0-736c-4d11-9ea4-3a7611ecc17f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.349994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b7d0-736c-4d11-9ea4-3a7611ecc17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.357773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:49.367522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7fd5b7d0-736c-4d11-9ea4-3a7611ecc17f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:49.368704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7fd5b7d0-736c-4d11-9ea4-3a7611ecc17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:49.409534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:49.417737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8e1a5-94d3-47e3-81c8-b8541a42bdc0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:49.437730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 527/1042] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:49.438486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5b7d0-736c-4d11-9ea4-3a7611ecc17f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:49.455746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f3c8a-783a-4873-9833-fdf1e0de5cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:49.456987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f3c8a-783a-4873-9833-fdf1e0de5cbe None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.457278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f3c8a-783a-4873-9833-fdf1e0de5cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.457563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f3c8a-783a-4873-9833-fdf1e0de5cbe None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.458312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f3c8a-783a-4873-9833-fdf1e0de5cbe None admin] RBAC: Adding query filter params (name=image) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:49.458624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f3c8a-783a-4873-9833-fdf1e0de5cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:49.460083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f3c8a-783a-4873-9833-fdf1e0de5cbe None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:49.465758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 516/1043] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:49.466388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f3c8a-783a-4873-9833-fdf1e0de5cbe None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:49.541134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5b7d0-736c-4d11-9ea4-3a7611ecc17f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:49.564248 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f3c8a-783a-4873-9833-fdf1e0de5cbe None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:49.566582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:49.576114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b8529-0622-4c40-bec8-f5dbed2474bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:49.576798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b8529-0622-4c40-bec8-f5dbed2474bd None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.577088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b8529-0622-4c40-bec8-f5dbed2474bd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.577364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b8529-0622-4c40-bec8-f5dbed2474bd None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.578096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b8529-0622-4c40-bec8-f5dbed2474bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:49.579557 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b8529-0622-4c40-bec8-f5dbed2474bd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:49.599036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57e383-2b85-4c03-9882-d1be751282ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:49.599846 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57e383-2b85-4c03-9882-d1be751282ea None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.600126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57e383-2b85-4c03-9882-d1be751282ea None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.600399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57e383-2b85-4c03-9882-d1be751282ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.600540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 528/1044] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:52:49 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:49.601333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b8529-0622-4c40-bec8-f5dbed2474bd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:49.620195 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57e383-2b85-4c03-9882-d1be751282ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-de57e383-2b85-4c03-9882-d1be751282ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:49.629075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:49.630538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:49.631824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 517/1045] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jun 20 10:52:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:49.638106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57e383-2b85-4c03-9882-d1be751282ea None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:49.653713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:49.739336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b8529-0622-4c40-bec8-f5dbed2474bd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:49.757453 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57e383-2b85-4c03-9882-d1be751282ea None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:49.777018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece5e68-cb06-4137-899a-7f5b88323722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:49.777704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece5e68-cb06-4137-899a-7f5b88323722 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.778007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece5e68-cb06-4137-899a-7f5b88323722 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.778329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece5e68-cb06-4137-899a-7f5b88323722 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.779376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece5e68-cb06-4137-899a-7f5b88323722 None admin] RBAC: Adding query filter params (type=image) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:49.779376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece5e68-cb06-4137-899a-7f5b88323722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:49.780733 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece5e68-cb06-4137-899a-7f5b88323722 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:49.787204 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 529/1046] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:49.793588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece5e68-cb06-4137-899a-7f5b88323722 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:49.796715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb7e7d-2a26-444a-a2d3-8b4c66a1d2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:49.797486 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb7e7d-2a26-444a-a2d3-8b4c66a1d2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.797747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb7e7d-2a26-444a-a2d3-8b4c66a1d2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.798066 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb7e7d-2a26-444a-a2d3-8b4c66a1d2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.798825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb7e7d-2a26-444a-a2d3-8b4c66a1d2a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:49.799135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb7e7d-2a26-444a-a2d3-8b4c66a1d2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:49.800516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb7e7d-2a26-444a-a2d3-8b4c66a1d2a0 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:49.809561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 518/1047] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jun 20 10:52:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:49.815726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb7e7d-2a26-444a-a2d3-8b4c66a1d2a0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:49.851555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:49.897265 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:49.905264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece5e68-cb06-4137-899a-7f5b88323722 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:49.938383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb7e7d-2a26-444a-a2d3-8b4c66a1d2a0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:49.940444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc311770-6117-4978-a63c-73fee69fcfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:49.941373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc311770-6117-4978-a63c-73fee69fcfb0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.941373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc311770-6117-4978-a63c-73fee69fcfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.941647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc311770-6117-4978-a63c-73fee69fcfb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.942442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc311770-6117-4978-a63c-73fee69fcfb0 None admin] RBAC: Adding query filter params (interface=public, service_id=96c8c8954cc84032bd9f8c3af75ec3db, region_id=RegionOne) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:49.942755 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc311770-6117-4978-a63c-73fee69fcfb0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:49.944119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc311770-6117-4978-a63c-73fee69fcfb0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:49.949591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 530/1048] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jun 20 10:52:49 2022] GET /identity/v3/endpoints?service_id=96c8c8954cc84032bd9f8c3af75ec3db&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:49.954891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc311770-6117-4978-a63c-73fee69fcfb0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:49.974536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432c95c-705d-469f-8def-e01d8e0b2e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:49.975337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432c95c-705d-469f-8def-e01d8e0b2e24 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:49.975617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432c95c-705d-469f-8def-e01d8e0b2e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:49.975887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432c95c-705d-469f-8def-e01d8e0b2e24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:49.987579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5432c95c-705d-469f-8def-e01d8e0b2e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:49.989036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5432c95c-705d-469f-8def-e01d8e0b2e24 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5432c95c-705d-469f-8def-e01d8e0b2e24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:50.007354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:50.008259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:50.009134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:50.009134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:50.009134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:50.009134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:50.009134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 519/1049] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:50.014414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432c95c-705d-469f-8def-e01d8e0b2e24 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:50.059237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc311770-6117-4978-a63c-73fee69fcfb0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:50.095566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91254275-0fd1-4e53-8185-380ce7908d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:50.096447 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91254275-0fd1-4e53-8185-380ce7908d6b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:50.096560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91254275-0fd1-4e53-8185-380ce7908d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:50.096760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91254275-0fd1-4e53-8185-380ce7908d6b None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:50.097222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:50.097652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91254275-0fd1-4e53-8185-380ce7908d6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:50.098901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91254275-0fd1-4e53-8185-380ce7908d6b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:50.107014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 531/1050] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jun 20 10:52:49 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 10:52:50.131593 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432c95c-705d-469f-8def-e01d8e0b2e24 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:50.171087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65377477-86ca-463b-b4e0-dc77a94fbcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:50.171776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65377477-86ca-463b-b4e0-dc77a94fbcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:50.172055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65377477-86ca-463b-b4e0-dc77a94fbcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:50.172327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65377477-86ca-463b-b4e0-dc77a94fbcb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:50.173060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65377477-86ca-463b-b4e0-dc77a94fbcb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:50.173370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65377477-86ca-463b-b4e0-dc77a94fbcb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:50.174825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65377477-86ca-463b-b4e0-dc77a94fbcb9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:50.189831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 520/1051] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:50.195582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91254275-0fd1-4e53-8185-380ce7908d6b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:50.223578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:50.307864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91254275-0fd1-4e53-8185-380ce7908d6b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:50.342037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdce8a-f4c5-43f1-97aa-ab1c09946aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:50.342858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdce8a-f4c5-43f1-97aa-ab1c09946aaf None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:50.343133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdce8a-f4c5-43f1-97aa-ab1c09946aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:50.343402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdce8a-f4c5-43f1-97aa-ab1c09946aaf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:50.351035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdce8a-f4c5-43f1-97aa-ab1c09946aaf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:50.352479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdce8a-f4c5-43f1-97aa-ab1c09946aaf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1afdce8a-f4c5-43f1-97aa-ab1c09946aaf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:50.367039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:50.367992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:50.369351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 532/1052] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jun 20 10:52:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:50.374592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65377477-86ca-463b-b4e0-dc77a94fbcb9 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:50.477022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65377477-86ca-463b-b4e0-dc77a94fbcb9 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:50.511011 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f484f-114d-43e5-99dc-6e335dace623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:50.511676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f484f-114d-43e5-99dc-6e335dace623 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:50.511954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f484f-114d-43e5-99dc-6e335dace623 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:50.512229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f484f-114d-43e5-99dc-6e335dace623 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:50.513066 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f484f-114d-43e5-99dc-6e335dace623 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:50.513367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f484f-114d-43e5-99dc-6e335dace623 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:50.514826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f484f-114d-43e5-99dc-6e335dace623 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:50.522755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:50.533806 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 521/1053] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jun 20 10:52:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:50.539255 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdce8a-f4c5-43f1-97aa-ab1c09946aaf None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:50.626153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:50.645734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdce8a-f4c5-43f1-97aa-ab1c09946aaf None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:50.680789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b11369-d949-43d4-9d58-fc35de080400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:50.681595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b11369-d949-43d4-9d58-fc35de080400 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:50.682045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b11369-d949-43d4-9d58-fc35de080400 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:50.682361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b11369-d949-43d4-9d58-fc35de080400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:50.693313 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b11369-d949-43d4-9d58-fc35de080400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:50.694812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b11369-d949-43d4-9d58-fc35de080400 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-05b11369-d949-43d4-9d58-fc35de080400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:50.712495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:50.713948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:50.714877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:50.714877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:50.714877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:50.714877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:50.714877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 533/1054] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:50.721181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f484f-114d-43e5-99dc-6e335dace623 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:50.830758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f484f-114d-43e5-99dc-6e335dace623 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:50.866895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a90929-e8dc-4d13-9586-c7ab148b022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:50.867569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90929-e8dc-4d13-9586-c7ab148b022c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:50.867849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90929-e8dc-4d13-9586-c7ab148b022c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:50.868128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90929-e8dc-4d13-9586-c7ab148b022c None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:50.868855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a90929-e8dc-4d13-9586-c7ab148b022c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:50.869163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a90929-e8dc-4d13-9586-c7ab148b022c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:50.870647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a90929-e8dc-4d13-9586-c7ab148b022c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:50.882192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 522/1055] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:50.888312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b11369-d949-43d4-9d58-fc35de080400 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:50.993875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b11369-d949-43d4-9d58-fc35de080400 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:51.028698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec2b85-2ff0-444e-bdcd-3faf418f5ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:51.029557 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec2b85-2ff0-444e-bdcd-3faf418f5ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:51.029837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec2b85-2ff0-444e-bdcd-3faf418f5ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:51.030193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec2b85-2ff0-444e-bdcd-3faf418f5ea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:51.032102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:51.037945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec2b85-2ff0-444e-bdcd-3faf418f5ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:51.039693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec2b85-2ff0-444e-bdcd-3faf418f5ea9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-b2ec2b85-2ff0-444e-bdcd-3faf418f5ea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:51.053960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:51.054855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:51.055746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:51.055746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:51.055746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:51.055746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:51.055746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:51.055746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:51.055746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:51.055746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:51.056761 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 534/1056] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:51.061291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a90929-e8dc-4d13-9586-c7ab148b022c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:51.122081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:51.169013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a90929-e8dc-4d13-9586-c7ab148b022c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:51.203854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dc524-a981-4dbb-8d72-16768dbbddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:51.204517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dc524-a981-4dbb-8d72-16768dbbddf9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:51.204796 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dc524-a981-4dbb-8d72-16768dbbddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:51.205077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dc524-a981-4dbb-8d72-16768dbbddf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:51.205853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dc524-a981-4dbb-8d72-16768dbbddf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:51.206185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dc524-a981-4dbb-8d72-16768dbbddf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:51.207991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dc524-a981-4dbb-8d72-16768dbbddf9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:51.221147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 523/1057] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:51.227173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec2b85-2ff0-444e-bdcd-3faf418f5ea9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:51.332921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec2b85-2ff0-444e-bdcd-3faf418f5ea9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:51.368659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0a787-4321-4300-8e0c-d2eda250b7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:51.370106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0a787-4321-4300-8e0c-d2eda250b7a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:51.370404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0a787-4321-4300-8e0c-d2eda250b7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:51.370675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0a787-4321-4300-8e0c-d2eda250b7a6 None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/37b1f13aa474493d83b3a16f76949fbc/roles/6db9b3d9148e4e37bded8945be0f6f6f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:51.372912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0a787-4321-4300-8e0c-d2eda250b7a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=37b1f13aa474493d83b3a16f76949fbc, role_id=6db9b3d9148e4e37bded8945be0f6f6f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:51.374370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0a787-4321-4300-8e0c-d2eda250b7a6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:51.388399 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 535/1058] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:52:51 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/37b1f13aa474493d83b3a16f76949fbc/roles/6db9b3d9148e4e37bded8945be0f6f6f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:52:51.707740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea66afc-ba3d-4021-b4e3-7912c7bf0488 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:51.708122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea66afc-ba3d-4021-b4e3-7912c7bf0488 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:51.708286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea66afc-ba3d-4021-b4e3-7912c7bf0488 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:51.709661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 524/1059] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:51.715436 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd778386-d3e7-4a6b-99f6-cde5d673c52e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:51.715682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd778386-d3e7-4a6b-99f6-cde5d673c52e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:51.715990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd778386-d3e7-4a6b-99f6-cde5d673c52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:51.733275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cd778386-d3e7-4a6b-99f6-cde5d673c52e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:51.734510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cd778386-d3e7-4a6b-99f6-cde5d673c52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:51.785720 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:51.805160 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 536/1060] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:51.828773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea66afc-ba3d-4021-b4e3-7912c7bf0488 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:51.941307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea66afc-ba3d-4021-b4e3-7912c7bf0488 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:51.959581 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:51.979268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487ec44-a64e-4594-bb2c-f59b267bf19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:51.980072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487ec44-a64e-4594-bb2c-f59b267bf19a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:51.980357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487ec44-a64e-4594-bb2c-f59b267bf19a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:51.980633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487ec44-a64e-4594-bb2c-f59b267bf19a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:51.991322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487ec44-a64e-4594-bb2c-f59b267bf19a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6487ec44-a64e-4594-bb2c-f59b267bf19a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:51.999333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:52.000292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:52.001150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 525/1061] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:52.006577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd778386-d3e7-4a6b-99f6-cde5d673c52e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:52.107805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd778386-d3e7-4a6b-99f6-cde5d673c52e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:52.142140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0794f-eae3-43c9-9fb5-5401746daed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:52.142837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0794f-eae3-43c9-9fb5-5401746daed9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:52.143119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0794f-eae3-43c9-9fb5-5401746daed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:52.143390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0794f-eae3-43c9-9fb5-5401746daed9 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:52.144137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0794f-eae3-43c9-9fb5-5401746daed9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:52.144443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0794f-eae3-43c9-9fb5-5401746daed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:52.145814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0794f-eae3-43c9-9fb5-5401746daed9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:52.152351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad312e2-3a16-4d4e-9dfb-9df3576fd4c9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:52.152829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad312e2-3a16-4d4e-9dfb-9df3576fd4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:52.153485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad312e2-3a16-4d4e-9dfb-9df3576fd4c9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:52.155065 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 526/1062] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:52.156369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 537/1063] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:52.160466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4bd92-ac8d-4d3c-a0f3-75bf6e92576f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:52.160738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4bd92-ac8d-4d3c-a0f3-75bf6e92576f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:52.161029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4bd92-ac8d-4d3c-a0f3-75bf6e92576f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:52.162190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0794f-eae3-43c9-9fb5-5401746daed9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:52.178073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-81f4bd92-ac8d-4d3c-a0f3-75bf6e92576f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:52.179285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-81f4bd92-ac8d-4d3c-a0f3-75bf6e92576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:52.211177 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:52.255492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 527/1064] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:52.270141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0794f-eae3-43c9-9fb5-5401746daed9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:52.278150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4bd92-ac8d-4d3c-a0f3-75bf6e92576f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:52.307572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ac670-414f-42ab-be02-12b35eeb2d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:52.308452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ac670-414f-42ab-be02-12b35eeb2d5b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:52.308741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ac670-414f-42ab-be02-12b35eeb2d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:52.309039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ac670-414f-42ab-be02-12b35eeb2d5b None admin] PATH_INFO: `/v3/users/bd8deb62810443f784032d822cb67b51` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:52.324680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ac670-414f-42ab-be02-12b35eeb2d5b None admin] RBAC: Authorizing `identity:get_user(user_id=bd8deb62810443f784032d822cb67b51)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:52.326176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ac670-414f-42ab-be02-12b35eeb2d5b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:52.328123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 538/1065] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:52:52 2022] GET /identity/v3/users/bd8deb62810443f784032d822cb67b51 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:52.334127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ac670-414f-42ab-be02-12b35eeb2d5b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:52.380213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4bd92-ac8d-4d3c-a0f3-75bf6e92576f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:52.389309 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:52.420081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed77d40-acc9-4850-b3ea-7b328250dbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:52.420995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed77d40-acc9-4850-b3ea-7b328250dbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:52.421412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed77d40-acc9-4850-b3ea-7b328250dbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:52.421681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed77d40-acc9-4850-b3ea-7b328250dbb6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:52.422376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed77d40-acc9-4850-b3ea-7b328250dbb6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:52.423776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed77d40-acc9-4850-b3ea-7b328250dbb6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-2ed77d40-acc9-4850-b3ea-7b328250dbb6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:52.439541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:52.440617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:52.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 528/1066] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Jun 20 10:52:52 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:52.442839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ac670-414f-42ab-be02-12b35eeb2d5b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:52.451095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed77d40-acc9-4850-b3ea-7b328250dbb6 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:52.459246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:52.507473 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956c1a6-6bf7-4ae1-b1af-7efad262283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:52.508412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956c1a6-6bf7-4ae1-b1af-7efad262283e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:52.508705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956c1a6-6bf7-4ae1-b1af-7efad262283e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:52.508986 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956c1a6-6bf7-4ae1-b1af-7efad262283e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:52.571580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed77d40-acc9-4850-b3ea-7b328250dbb6 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:52.580163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956c1a6-6bf7-4ae1-b1af-7efad262283e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:52.582110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956c1a6-6bf7-4ae1-b1af-7efad262283e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:52.609266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983973d-c1e4-4ad2-bd67-e92461567a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:52.610294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983973d-c1e4-4ad2-bd67-e92461567a0b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:52.610726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983973d-c1e4-4ad2-bd67-e92461567a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:52.611004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983973d-c1e4-4ad2-bd67-e92461567a0b None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:52.611723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983973d-c1e4-4ad2-bd67-e92461567a0b None admin] RBAC: Adding query filter params (name=image) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:52.612035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983973d-c1e4-4ad2-bd67-e92461567a0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:52.613429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983973d-c1e4-4ad2-bd67-e92461567a0b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:52.618974 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 529/1067] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:52.624743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983973d-c1e4-4ad2-bd67-e92461567a0b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:52.660356 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:52.702662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 539/1068] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:52 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 10:52:52.703798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956c1a6-6bf7-4ae1-b1af-7efad262283e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:52.707637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:52.736999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983973d-c1e4-4ad2-bd67-e92461567a0b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:52.775872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d20d7-7038-4fb6-964d-f48c36647239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:52.777556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d20d7-7038-4fb6-964d-f48c36647239 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:52.777965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d20d7-7038-4fb6-964d-f48c36647239 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:52.778392 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d20d7-7038-4fb6-964d-f48c36647239 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:52.779428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d20d7-7038-4fb6-964d-f48c36647239 None admin] RBAC: Adding query filter params (type=image) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:52.779851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d20d7-7038-4fb6-964d-f48c36647239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:52.782228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d20d7-7038-4fb6-964d-f48c36647239 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:52.794348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 530/1069] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:52.795479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d20d7-7038-4fb6-964d-f48c36647239 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:52.811019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956c1a6-6bf7-4ae1-b1af-7efad262283e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:52.850776 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42e15f-4531-47f5-873b-5073bb9c1121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:52.851679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42e15f-4531-47f5-873b-5073bb9c1121 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:52.851953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42e15f-4531-47f5-873b-5073bb9c1121 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:52.852226 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42e15f-4531-47f5-873b-5073bb9c1121 None admin] PATH_INFO: `/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:52.853403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42e15f-4531-47f5-873b-5073bb9c1121 None admin] RBAC: Authorizing `identity:get_project(project_id=3fd4b4a1abd349b2ac625c514aff32c1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:52.855416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42e15f-4531-47f5-873b-5073bb9c1121 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:52.857130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 540/1070] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:52:52 2022] GET /identity/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:52.863806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42e15f-4531-47f5-873b-5073bb9c1121 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:52.917984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:52.921637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d20d7-7038-4fb6-964d-f48c36647239 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:52.939326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:52.964145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057e386-4c03-4c7f-9bd8-a5a9eccf2d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:52.965134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057e386-4c03-4c7f-9bd8-a5a9eccf2d15 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:52.965421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057e386-4c03-4c7f-9bd8-a5a9eccf2d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:52.965758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057e386-4c03-4c7f-9bd8-a5a9eccf2d15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:52.966550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057e386-4c03-4c7f-9bd8-a5a9eccf2d15 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:52.967942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057e386-4c03-4c7f-9bd8-a5a9eccf2d15 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:52.973408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42e15f-4531-47f5-873b-5073bb9c1121 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:52.998489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 531/1071] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jun 20 10:52:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:52:53.022690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da902c-b8c8-43f1-83fc-c000c76141c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:53.023366 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da902c-b8c8-43f1-83fc-c000c76141c2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:53.023647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da902c-b8c8-43f1-83fc-c000c76141c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:53.023927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da902c-b8c8-43f1-83fc-c000c76141c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:53.024763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da902c-b8c8-43f1-83fc-c000c76141c2 None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=3fd4b4a1abd349b2ac625c514aff32c1, user.id=bd8deb62810443f784032d822cb67b51) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:53.025076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da902c-b8c8-43f1-83fc-c000c76141c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:53.026644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da902c-b8c8-43f1-83fc-c000c76141c2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:53.033261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 541/1072] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:52 2022] GET /identity/v3/role_assignments?user.id=bd8deb62810443f784032d822cb67b51&scope.project.id=3fd4b4a1abd349b2ac625c514aff32c1&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:53.411686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4be5b-dd1a-418d-9799-f19318bb7ae2 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:53.412183 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4be5b-dd1a-418d-9799-f19318bb7ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:53.412261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4be5b-dd1a-418d-9799-f19318bb7ae2 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:53.413633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 532/1073] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:53.419035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3c8dd-627f-46f5-8fe7-de4cbe8986d9 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:53.419445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3c8dd-627f-46f5-8fe7-de4cbe8986d9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:53.419633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3c8dd-627f-46f5-8fe7-de4cbe8986d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:53.437477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8ca3c8dd-627f-46f5-8fe7-de4cbe8986d9 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:53.438704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8ca3c8dd-627f-46f5-8fe7-de4cbe8986d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:53.512882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 542/1074] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:53.536179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4be5b-dd1a-418d-9799-f19318bb7ae2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:53.642065 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd4be5b-dd1a-418d-9799-f19318bb7ae2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:53.676202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce43c3-631c-4e16-80ed-b221b6650ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:53.676943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce43c3-631c-4e16-80ed-b221b6650ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:53.677223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce43c3-631c-4e16-80ed-b221b6650ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:53.677497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce43c3-631c-4e16-80ed-b221b6650ec8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:53.688632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce43c3-631c-4e16-80ed-b221b6650ec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-7fce43c3-631c-4e16-80ed-b221b6650ec8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:53.696723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:53.697701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 10:52:53.698922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:53.702478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 533/1075] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Jun 20 10:52:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:53.704419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3c8dd-627f-46f5-8fe7-de4cbe8986d9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:53.772423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:53.812042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3c8dd-627f-46f5-8fe7-de4cbe8986d9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:53.845931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f739c-1c28-4b4e-8505-c4749d3a7333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:53.846637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f739c-1c28-4b4e-8505-c4749d3a7333 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:53.846922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f739c-1c28-4b4e-8505-c4749d3a7333 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:53.847195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f739c-1c28-4b4e-8505-c4749d3a7333 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:53.847949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f739c-1c28-4b4e-8505-c4749d3a7333 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:53.848264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f739c-1c28-4b4e-8505-c4749d3a7333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:53.849646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f739c-1c28-4b4e-8505-c4749d3a7333 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:53.862237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 543/1076] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jun 20 10:52:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:53.863773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce43c3-631c-4e16-80ed-b221b6650ec8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:53.879852 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:53.982296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce43c3-631c-4e16-80ed-b221b6650ec8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:54.017939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c700670-bb2f-43ae-8daa-80952960e52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:54.018772 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c700670-bb2f-43ae-8daa-80952960e52d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:54.019060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c700670-bb2f-43ae-8daa-80952960e52d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:54.019337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c700670-bb2f-43ae-8daa-80952960e52d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:54.030188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c700670-bb2f-43ae-8daa-80952960e52d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:54.031661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c700670-bb2f-43ae-8daa-80952960e52d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6c700670-bb2f-43ae-8daa-80952960e52d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:54.049722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:54.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:54.056437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:54.056437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:54.056437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:54.056437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:54.056437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 534/1077] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:54.058291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f739c-1c28-4b4e-8505-c4749d3a7333 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:54.165950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f739c-1c28-4b4e-8505-c4749d3a7333 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:54.189587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:54.201941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5216a8-75a9-4617-9dd4-737855f00efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:54.202692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5216a8-75a9-4617-9dd4-737855f00efc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:54.202981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5216a8-75a9-4617-9dd4-737855f00efc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:54.203259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5216a8-75a9-4617-9dd4-737855f00efc None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:54.204034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5216a8-75a9-4617-9dd4-737855f00efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:54.204354 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5216a8-75a9-4617-9dd4-737855f00efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:54.205737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5216a8-75a9-4617-9dd4-737855f00efc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:54.219741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 544/1078] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:54.225206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c700670-bb2f-43ae-8daa-80952960e52d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:54.293878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:54.331562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c700670-bb2f-43ae-8daa-80952960e52d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:54.366545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce0439-d2a7-4c0a-a8f7-902e34205ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:54.367336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce0439-d2a7-4c0a-a8f7-902e34205ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:54.367624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce0439-d2a7-4c0a-a8f7-902e34205ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:54.367898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce0439-d2a7-4c0a-a8f7-902e34205ae9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:54.375168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce0439-d2a7-4c0a-a8f7-902e34205ae9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:54.376633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce0439-d2a7-4c0a-a8f7-902e34205ae9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-b8ce0439-d2a7-4c0a-a8f7-902e34205ae9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:52:54.391293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:54.392225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:54.398384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 535/1079] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Jun 20 10:52:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:54.400348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5216a8-75a9-4617-9dd4-737855f00efc None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:54.507122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5216a8-75a9-4617-9dd4-737855f00efc None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:54.543263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab710d-9e15-488a-94c0-967b2cf66f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:54.544002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab710d-9e15-488a-94c0-967b2cf66f8d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:54.544218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab710d-9e15-488a-94c0-967b2cf66f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:54.544497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab710d-9e15-488a-94c0-967b2cf66f8d None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:54.545337 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab710d-9e15-488a-94c0-967b2cf66f8d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:54.545654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab710d-9e15-488a-94c0-967b2cf66f8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:54.547141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab710d-9e15-488a-94c0-967b2cf66f8d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:54.570148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 545/1080] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Jun 20 10:52:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:54.571783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce0439-d2a7-4c0a-a8f7-902e34205ae9 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:54.677192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce0439-d2a7-4c0a-a8f7-902e34205ae9 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:54.704191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:54.714994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc47060-6ccf-412d-9aa8-d92cb384207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:54.715853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47060-6ccf-412d-9aa8-d92cb384207b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:54.716308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47060-6ccf-412d-9aa8-d92cb384207b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:54.716729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47060-6ccf-412d-9aa8-d92cb384207b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:54.789230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc47060-6ccf-412d-9aa8-d92cb384207b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:54.791022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc47060-6ccf-412d-9aa8-d92cb384207b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:54.897177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 536/1081] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:54.905486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab710d-9e15-488a-94c0-967b2cf66f8d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:54.952982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:55.016584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab710d-9e15-488a-94c0-967b2cf66f8d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:55.051912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d013a6-4a71-4878-84b9-f5e52e368980 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.052241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d013a6-4a71-4878-84b9-f5e52e368980 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.052454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d013a6-4a71-4878-84b9-f5e52e368980 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.053099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18618028-874c-4699-b931-61fcdd321be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:55.053998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 537/1082] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:52:55.054246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18618028-874c-4699-b931-61fcdd321be7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.054246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18618028-874c-4699-b931-61fcdd321be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.054465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18618028-874c-4699-b931-61fcdd321be7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.058792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a3565-afcd-42f6-a4dc-a9e174b370a3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.059079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a3565-afcd-42f6-a4dc-a9e174b370a3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.059367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a3565-afcd-42f6-a4dc-a9e174b370a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.065247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18618028-874c-4699-b931-61fcdd321be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:55.066736 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18618028-874c-4699-b931-61fcdd321be7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:55.076214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9c4a3565-afcd-42f6-a4dc-a9e174b370a3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:55.077408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9c4a3565-afcd-42f6-a4dc-a9e174b370a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-18618028-874c-4699-b931-61fcdd321be7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:55.084843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:55.085878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:55.086810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:55.086810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:55.086810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:55.086810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:55.086810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 546/1083] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:52:55.087412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e354d67-b5c2-476e-ad72-22c5de518a3c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.087614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e354d67-b5c2-476e-ad72-22c5de518a3c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.087928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e354d67-b5c2-476e-ad72-22c5de518a3c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.089639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 547/1084] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:55.091832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e354d67-b5c2-476e-ad72-22c5de518a3c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:55.099427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:55.150169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 538/1085] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:52:55.150169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2abdb-9585-47aa-893f-8100d2f6d00c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.150169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2abdb-9585-47aa-893f-8100d2f6d00c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.150600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2abdb-9585-47aa-893f-8100d2f6d00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.165846 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c0a2abdb-9585-47aa-893f-8100d2f6d00c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:55.167322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c0a2abdb-9585-47aa-893f-8100d2f6d00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:55.185990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:55.194327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e354d67-b5c2-476e-ad72-22c5de518a3c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:55.228546 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16c2fc-c609-4581-9684-d6be5b778b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:55.229214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16c2fc-c609-4581-9684-d6be5b778b2d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.229516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16c2fc-c609-4581-9684-d6be5b778b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.229791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16c2fc-c609-4581-9684-d6be5b778b2d None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.230538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16c2fc-c609-4581-9684-d6be5b778b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:55.230846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16c2fc-c609-4581-9684-d6be5b778b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:55.231750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 539/1086] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:55.232224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16c2fc-c609-4581-9684-d6be5b778b2d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:55.232573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2abdb-9585-47aa-893f-8100d2f6d00c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:55.248883 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 548/1087] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:55.254383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16c2fc-c609-4581-9684-d6be5b778b2d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:55.319640 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:55.334476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2abdb-9585-47aa-893f-8100d2f6d00c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:55.352556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16c2fc-c609-4581-9684-d6be5b778b2d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:55.368220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d807f0-0b76-4986-9bf1-f29cf6c06618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:55.368882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d807f0-0b76-4986-9bf1-f29cf6c06618 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.369264 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d807f0-0b76-4986-9bf1-f29cf6c06618 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.369548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d807f0-0b76-4986-9bf1-f29cf6c06618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.386223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801031c-06a1-48d7-8eb0-06ac69e835dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:55.387022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801031c-06a1-48d7-8eb0-06ac69e835dd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.387311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801031c-06a1-48d7-8eb0-06ac69e835dd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.387590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801031c-06a1-48d7-8eb0-06ac69e835dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.394162 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801031c-06a1-48d7-8eb0-06ac69e835dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:55.395651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801031c-06a1-48d7-8eb0-06ac69e835dd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-3801031c-06a1-48d7-8eb0-06ac69e835dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:52:55.409722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:55.410715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:55.411600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:52:55.411600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:52:55.411600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:52:55.411600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:52:55.411600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:52:55.411600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:52:55.411600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:52:55.411600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:55.411600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 549/1088] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:52:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:55.412470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801031c-06a1-48d7-8eb0-06ac69e835dd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:55.426582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:55.442200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d807f0-0b76-4986-9bf1-f29cf6c06618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:55.443689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d807f0-0b76-4986-9bf1-f29cf6c06618 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:55.509089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801031c-06a1-48d7-8eb0-06ac69e835dd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:55.527527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:55.542636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37041aff-3d21-4bbb-89f0-d354ec079dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:55.543429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37041aff-3d21-4bbb-89f0-d354ec079dea None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.543570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 540/1089] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:55.543745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37041aff-3d21-4bbb-89f0-d354ec079dea None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.543970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37041aff-3d21-4bbb-89f0-d354ec079dea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.544296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d807f0-0b76-4986-9bf1-f29cf6c06618 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:55.558563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37041aff-3d21-4bbb-89f0-d354ec079dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-37041aff-3d21-4bbb-89f0-d354ec079dea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:55.566027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:55.567184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:55.568054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 550/1090] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:55.568781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37041aff-3d21-4bbb-89f0-d354ec079dea None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:55.642211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d807f0-0b76-4986-9bf1-f29cf6c06618 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:55.643970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:55.669179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37041aff-3d21-4bbb-89f0-d354ec079dea None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:55.674931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b62c1-d94b-43c6-86b9-f9268d8341ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:55.675591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b62c1-d94b-43c6-86b9-f9268d8341ed None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.675895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b62c1-d94b-43c6-86b9-f9268d8341ed None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.676157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b62c1-d94b-43c6-86b9-f9268d8341ed None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.676942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b62c1-d94b-43c6-86b9-f9268d8341ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:55.677250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b62c1-d94b-43c6-86b9-f9268d8341ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:55.678723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b62c1-d94b-43c6-86b9-f9268d8341ed None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:55.690132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 541/1091] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:52:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:55.690910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b62c1-d94b-43c6-86b9-f9268d8341ed None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:55.702497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a50495-6488-4363-a2cf-d550d4f66aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:55.703301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a50495-6488-4363-a2cf-d550d4f66aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.703607 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a50495-6488-4363-a2cf-d550d4f66aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.703879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a50495-6488-4363-a2cf-d550d4f66aa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.714866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a50495-6488-4363-a2cf-d550d4f66aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:55.716305 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a50495-6488-4363-a2cf-d550d4f66aa8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e7a50495-6488-4363-a2cf-d550d4f66aa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:52:55.735080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:55.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:55.737022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:52:55.737022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:52:55.737022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:52:55.737022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:52:55.737022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 551/1092] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:52:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:55.737295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a50495-6488-4363-a2cf-d550d4f66aa8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:55.742402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:55.796517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b62c1-d94b-43c6-86b9-f9268d8341ed None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:55.828933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a351667-b2aa-4afd-888a-d801e577f9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:55.829629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a351667-b2aa-4afd-888a-d801e577f9ae None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.829912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a351667-b2aa-4afd-888a-d801e577f9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.830235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a351667-b2aa-4afd-888a-d801e577f9ae None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.830984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a351667-b2aa-4afd-888a-d801e577f9ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:55.831653 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a351667-b2aa-4afd-888a-d801e577f9ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:55.833043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a351667-b2aa-4afd-888a-d801e577f9ae None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:55.837240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a50495-6488-4363-a2cf-d550d4f66aa8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:55.842740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 542/1093] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:55.843349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a351667-b2aa-4afd-888a-d801e577f9ae None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:55.862636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:55.872266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53833acc-75a6-4986-a9f5-53635260bd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:55.872941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53833acc-75a6-4986-a9f5-53635260bd04 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:55.873225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53833acc-75a6-4986-a9f5-53635260bd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:55.873504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53833acc-75a6-4986-a9f5-53635260bd04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:55.874363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53833acc-75a6-4986-a9f5-53635260bd04 None admin] RBAC: Adding query filter params (role.id=6db9b3d9148e4e37bded8945be0f6f6f, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=37b1f13aa474493d83b3a16f76949fbc) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:55.874679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53833acc-75a6-4986-a9f5-53635260bd04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:55.876143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53833acc-75a6-4986-a9f5-53635260bd04 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:55.881920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 552/1094] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:55 2022] GET /identity/v3/role_assignments?user.id=37b1f13aa474493d83b3a16f76949fbc&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=6db9b3d9148e4e37bded8945be0f6f6f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:55.882728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53833acc-75a6-4986-a9f5-53635260bd04 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:55.967854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a351667-b2aa-4afd-888a-d801e577f9ae None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:55.978721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:56.004634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53833acc-75a6-4986-a9f5-53635260bd04 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:56.007215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68872947-2a7e-4c81-868f-e1f08d26a000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:56.008246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68872947-2a7e-4c81-868f-e1f08d26a000 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:56.008246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68872947-2a7e-4c81-868f-e1f08d26a000 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:56.008628 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68872947-2a7e-4c81-868f-e1f08d26a000 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:56.009437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68872947-2a7e-4c81-868f-e1f08d26a000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:56.009867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68872947-2a7e-4c81-868f-e1f08d26a000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:56.011407 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68872947-2a7e-4c81-868f-e1f08d26a000 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:56.034113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 543/1095] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:56.056208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e6933-3d65-47ee-b11c-c2b8a8927358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:56.057181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e6933-3d65-47ee-b11c-c2b8a8927358 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:56.057464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e6933-3d65-47ee-b11c-c2b8a8927358 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:56.057743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e6933-3d65-47ee-b11c-c2b8a8927358 None admin] PATH_INFO: `/v3/users/bd8deb62810443f784032d822cb67b51` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:56.059117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e6933-3d65-47ee-b11c-c2b8a8927358 None admin] RBAC: Authorizing `identity:get_user(user_id=bd8deb62810443f784032d822cb67b51)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:56.060621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e6933-3d65-47ee-b11c-c2b8a8927358 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:56.062445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 553/1096] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:52:55 2022] GET /identity/v3/users/bd8deb62810443f784032d822cb67b51 => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:56.068458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68872947-2a7e-4c81-868f-e1f08d26a000 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:56.184113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68872947-2a7e-4c81-868f-e1f08d26a000 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:56.222227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa64fc-15a8-4262-965d-a82c2a18a102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:56.223043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa64fc-15a8-4262-965d-a82c2a18a102 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:56.223326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa64fc-15a8-4262-965d-a82c2a18a102 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:56.223607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa64fc-15a8-4262-965d-a82c2a18a102 None admin] PATH_INFO: `/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:56.224803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa64fc-15a8-4262-965d-a82c2a18a102 None admin] RBAC: Authorizing `identity:get_project(project_id=3fd4b4a1abd349b2ac625c514aff32c1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:56.226330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa64fc-15a8-4262-965d-a82c2a18a102 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:56.228381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 544/1097] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:52:56 2022] GET /identity/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:56.234714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e6933-3d65-47ee-b11c-c2b8a8927358 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:56.298524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:56.344561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e6933-3d65-47ee-b11c-c2b8a8927358 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:56.378626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd697e9f-1471-46b5-89dd-6b307bdc34c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:56.379525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697e9f-1471-46b5-89dd-6b307bdc34c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:56.379805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697e9f-1471-46b5-89dd-6b307bdc34c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:56.380076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697e9f-1471-46b5-89dd-6b307bdc34c4 None admin] PATH_INFO: `/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1/users/bd8deb62810443f784032d822cb67b51/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:56.381996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd697e9f-1471-46b5-89dd-6b307bdc34c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fd4b4a1abd349b2ac625c514aff32c1, user_id=bd8deb62810443f784032d822cb67b51, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:56.383487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd697e9f-1471-46b5-89dd-6b307bdc34c4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:56.399888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 554/1098] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:52:56 2022] PUT /identity/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1/users/bd8deb62810443f784032d822cb67b51/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:52:58.375447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94d514-3a2b-4323-b1bd-cecc96f52189 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:58.375992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94d514-3a2b-4323-b1bd-cecc96f52189 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:58.376082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94d514-3a2b-4323-b1bd-cecc96f52189 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:58.378185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 545/1099] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:52:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:52:58.384464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89879e-d8c2-4927-984d-52cd5bc5ff72 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:58.385257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89879e-d8c2-4927-984d-52cd5bc5ff72 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:58.385257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89879e-d8c2-4927-984d-52cd5bc5ff72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:58.408532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cb89879e-d8c2-4927-984d-52cd5bc5ff72 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:52:58.409727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cb89879e-d8c2-4927-984d-52cd5bc5ff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:52:58.482050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 555/1100] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:52:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:52:58.505640 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94d514-3a2b-4323-b1bd-cecc96f52189 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:58.545490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:58.617288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94d514-3a2b-4323-b1bd-cecc96f52189 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:58.653033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fc838-6b44-420a-8a40-6c89abc824e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:58.653761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fc838-6b44-420a-8a40-6c89abc824e1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:58.654042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fc838-6b44-420a-8a40-6c89abc824e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:58.654350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fc838-6b44-420a-8a40-6c89abc824e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:58.665326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fc838-6b44-420a-8a40-6c89abc824e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-d80fc838-6b44-420a-8a40-6c89abc824e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:52:58.672920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:52:58.674015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:52:58.674875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 546/1101] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:52:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:52:58.680421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89879e-d8c2-4927-984d-52cd5bc5ff72 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:58.736801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:58.792759 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89879e-d8c2-4927-984d-52cd5bc5ff72 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:58.827751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710e9d9-0ff2-4b0d-a6e2-690dbe992bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:58.828407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710e9d9-0ff2-4b0d-a6e2-690dbe992bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:58.828690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710e9d9-0ff2-4b0d-a6e2-690dbe992bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:58.828969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710e9d9-0ff2-4b0d-a6e2-690dbe992bb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:58.829732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710e9d9-0ff2-4b0d-a6e2-690dbe992bb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:58.830043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710e9d9-0ff2-4b0d-a6e2-690dbe992bb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:58.831831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710e9d9-0ff2-4b0d-a6e2-690dbe992bb6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:58.840360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 556/1102] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:52:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:52:58.848897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fc838-6b44-420a-8a40-6c89abc824e1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:58.953316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:58.960532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fc838-6b44-420a-8a40-6c89abc824e1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:58.996607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d88f5-9511-47e1-96ec-fed4f02d14e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:58.997476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d88f5-9511-47e1-96ec-fed4f02d14e2 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:58.997752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d88f5-9511-47e1-96ec-fed4f02d14e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:58.998028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d88f5-9511-47e1-96ec-fed4f02d14e2 None admin] PATH_INFO: `/v3/users/bd8deb62810443f784032d822cb67b51` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:58.999376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d88f5-9511-47e1-96ec-fed4f02d14e2 None admin] RBAC: Authorizing `identity:get_user(user_id=bd8deb62810443f784032d822cb67b51)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:59.000833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d88f5-9511-47e1-96ec-fed4f02d14e2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:59.002519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 547/1103] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:52:58 2022] GET /identity/v3/users/bd8deb62810443f784032d822cb67b51 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:59.008773 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710e9d9-0ff2-4b0d-a6e2-690dbe992bb6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:59.115220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710e9d9-0ff2-4b0d-a6e2-690dbe992bb6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:59.126564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:59.155847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc13b66-ff26-4a49-bcba-4a8d62f2bf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:59.156488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc13b66-ff26-4a49-bcba-4a8d62f2bf4e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:59.156769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc13b66-ff26-4a49-bcba-4a8d62f2bf4e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:59.157040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc13b66-ff26-4a49-bcba-4a8d62f2bf4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:59.237014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc13b66-ff26-4a49-bcba-4a8d62f2bf4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:59.238411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc13b66-ff26-4a49-bcba-4a8d62f2bf4e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:59.353869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 557/1104] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:52:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:52:59.363300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d88f5-9511-47e1-96ec-fed4f02d14e2 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:59.474269 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d88f5-9511-47e1-96ec-fed4f02d14e2 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:59.509201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8ce05-689e-493c-9376-e01f9132c8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:59.510216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8ce05-689e-493c-9376-e01f9132c8ca None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:59.510602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8ce05-689e-493c-9376-e01f9132c8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:59.510879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8ce05-689e-493c-9376-e01f9132c8ca None admin] PATH_INFO: `/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:59.512180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8ce05-689e-493c-9376-e01f9132c8ca None admin] RBAC: Authorizing `identity:get_project(project_id=3fd4b4a1abd349b2ac625c514aff32c1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:59.513621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8ce05-689e-493c-9376-e01f9132c8ca None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:59.515327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 548/1105] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:52:59 2022] GET /identity/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:52:59.522303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc13b66-ff26-4a49-bcba-4a8d62f2bf4e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:52:59.540057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:52:59.651085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc13b66-ff26-4a49-bcba-4a8d62f2bf4e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:52:59.685915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f7d86-9182-41b3-90e2-462938f4964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:52:59.686595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f7d86-9182-41b3-90e2-462938f4964d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:52:59.686875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f7d86-9182-41b3-90e2-462938f4964d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:52:59.687143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f7d86-9182-41b3-90e2-462938f4964d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:52:59.687963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f7d86-9182-41b3-90e2-462938f4964d None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=3fd4b4a1abd349b2ac625c514aff32c1, user.id=bd8deb62810443f784032d822cb67b51) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:52:59.688279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f7d86-9182-41b3-90e2-462938f4964d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:52:59.689702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f7d86-9182-41b3-90e2-462938f4964d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:52:59.695973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 558/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:52:59 2022] GET /identity/v3/role_assignments?user.id=bd8deb62810443f784032d822cb67b51&scope.project.id=3fd4b4a1abd349b2ac625c514aff32c1&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:01.746648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6ef3-2e0e-42b0-854d-aaa6860c075a None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:01.747665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6ef3-2e0e-42b0-854d-aaa6860c075a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:01.747665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6ef3-2e0e-42b0-854d-aaa6860c075a None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:01.749762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 549/1107] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:01.758432 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62f01-f932-4b1e-a05c-84714d38797f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:01.759159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62f01-f932-4b1e-a05c-84714d38797f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:01.759321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62f01-f932-4b1e-a05c-84714d38797f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:01.778069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bab62f01-f932-4b1e-a05c-84714d38797f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:01.779272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bab62f01-f932-4b1e-a05c-84714d38797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:01.850427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 559/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:53:01.876305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f6ef3-2e0e-42b0-854d-aaa6860c075a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:01.917955 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:01.998953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f6ef3-2e0e-42b0-854d-aaa6860c075a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:02.037695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941140bb-45f3-4fc7-8755-08133e7a5a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:02.038519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941140bb-45f3-4fc7-8755-08133e7a5a27 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:02.038798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941140bb-45f3-4fc7-8755-08133e7a5a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:02.039072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941140bb-45f3-4fc7-8755-08133e7a5a27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:02.040354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941140bb-45f3-4fc7-8755-08133e7a5a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:02.041880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941140bb-45f3-4fc7-8755-08133e7a5a27 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:02.043561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 550/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:53:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:02.049848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab62f01-f932-4b1e-a05c-84714d38797f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:02.053466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:02.157341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab62f01-f932-4b1e-a05c-84714d38797f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:02.192894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7634e8-3ae1-4ecd-93d0-d30be709de07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:02.193555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7634e8-3ae1-4ecd-93d0-d30be709de07 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:02.193829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7634e8-3ae1-4ecd-93d0-d30be709de07 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:02.194136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7634e8-3ae1-4ecd-93d0-d30be709de07 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:02.194832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7634e8-3ae1-4ecd-93d0-d30be709de07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:02.196276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7634e8-3ae1-4ecd-93d0-d30be709de07 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:02.221045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 560/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:53:02 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:53:04.273867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50786c90-b36b-4f49-9b1a-a1d178c0e58a None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:04.274788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50786c90-b36b-4f49-9b1a-a1d178c0e58a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:04.274788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50786c90-b36b-4f49-9b1a-a1d178c0e58a None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:04.276565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 551/1111] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:04.283204 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20956e36-fce7-4ae3-bb8c-27fda6dcc32b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:04.283948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20956e36-fce7-4ae3-bb8c-27fda6dcc32b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:04.284133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20956e36-fce7-4ae3-bb8c-27fda6dcc32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:04.308490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-20956e36-fce7-4ae3-bb8c-27fda6dcc32b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:04.310121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-20956e36-fce7-4ae3-bb8c-27fda6dcc32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:04.370386 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:04.388937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 561/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:04.421330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50786c90-b36b-4f49-9b1a-a1d178c0e58a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:04.528746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:04.534063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50786c90-b36b-4f49-9b1a-a1d178c0e58a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:04.572479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a66da-df44-4d62-9313-94e8be4b62ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:04.573352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a66da-df44-4d62-9313-94e8be4b62ba None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:04.573641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a66da-df44-4d62-9313-94e8be4b62ba None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:04.573919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a66da-df44-4d62-9313-94e8be4b62ba None admin] PATH_INFO: `/v3/roles/ffab93457c0f4f64a29764f17759396e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:04.575505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a66da-df44-4d62-9313-94e8be4b62ba None admin] RBAC: Authorizing `identity:get_role(role_id=ffab93457c0f4f64a29764f17759396e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:04.576934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a66da-df44-4d62-9313-94e8be4b62ba None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:04.578417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 552/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:04 2022] GET /identity/v3/roles/ffab93457c0f4f64a29764f17759396e => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:04.585009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20956e36-fce7-4ae3-bb8c-27fda6dcc32b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:04.720988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20956e36-fce7-4ae3-bb8c-27fda6dcc32b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:04.758796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e73b3-58de-4043-87d4-1a47189b3061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:04.759748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e73b3-58de-4043-87d4-1a47189b3061 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:04.760025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e73b3-58de-4043-87d4-1a47189b3061 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:04.760297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e73b3-58de-4043-87d4-1a47189b3061 None admin] PATH_INFO: `/v3/users/979650d6056c4d71b67c35d96e206589` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:04.775300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e73b3-58de-4043-87d4-1a47189b3061 None admin] RBAC: Authorizing `identity:get_user(user_id=979650d6056c4d71b67c35d96e206589)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:04.776903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e73b3-58de-4043-87d4-1a47189b3061 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:04.778605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 562/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:04 2022] GET /identity/v3/users/979650d6056c4d71b67c35d96e206589 => generated 286 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:04.788193 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a66da-df44-4d62-9313-94e8be4b62ba None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:04.927859 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a66da-df44-4d62-9313-94e8be4b62ba None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:04.971259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14ea16-1e6a-45ec-98db-6235d535b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:04.971906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14ea16-1e6a-45ec-98db-6235d535b534 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:04.972193 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14ea16-1e6a-45ec-98db-6235d535b534 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:04.972467 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14ea16-1e6a-45ec-98db-6235d535b534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:05.009952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:05.052000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14ea16-1e6a-45ec-98db-6235d535b534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:05.053392 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14ea16-1e6a-45ec-98db-6235d535b534 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:05.151428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 553/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:53:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:53:05.160507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e73b3-58de-4043-87d4-1a47189b3061 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:05.212755 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:05.269525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e73b3-58de-4043-87d4-1a47189b3061 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:05.304296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5616d-1a86-43d3-87ca-4cb42a1d6a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:05.305094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5616d-1a86-43d3-87ca-4cb42a1d6a23 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:05.305368 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5616d-1a86-43d3-87ca-4cb42a1d6a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:05.305609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5616d-1a86-43d3-87ca-4cb42a1d6a23 None admin] PATH_INFO: `/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:05.306940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5616d-1a86-43d3-87ca-4cb42a1d6a23 None admin] RBAC: Authorizing `identity:get_project(project_id=3fd4b4a1abd349b2ac625c514aff32c1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:05.308457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5616d-1a86-43d3-87ca-4cb42a1d6a23 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:05.310241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 563/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:53:05 2022] GET /identity/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:53:05.319497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14ea16-1e6a-45ec-98db-6235d535b534 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:05.390791 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:05.424559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14ea16-1e6a-45ec-98db-6235d535b534 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:05.468661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12354a4-3eef-4480-a4d0-0a3b64235bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:05.469130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12354a4-3eef-4480-a4d0-0a3b64235bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:05.469387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12354a4-3eef-4480-a4d0-0a3b64235bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:05.469684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12354a4-3eef-4480-a4d0-0a3b64235bd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:05.470537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12354a4-3eef-4480-a4d0-0a3b64235bd8 None admin] RBAC: Adding query filter params (role.id=ffab93457c0f4f64a29764f17759396e, scope.project.id=3fd4b4a1abd349b2ac625c514aff32c1, user.id=979650d6056c4d71b67c35d96e206589) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:05.470850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12354a4-3eef-4480-a4d0-0a3b64235bd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:05.472302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12354a4-3eef-4480-a4d0-0a3b64235bd8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:05.478173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 554/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:53:05 2022] GET /identity/v3/role_assignments?user.id=979650d6056c4d71b67c35d96e206589&scope.project.id=3fd4b4a1abd349b2ac625c514aff32c1&role.id=ffab93457c0f4f64a29764f17759396e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:07.438557 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b780ed-030f-4fd1-b9d6-b0564c3bad18 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:07.439145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b780ed-030f-4fd1-b9d6-b0564c3bad18 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:07.439145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b780ed-030f-4fd1-b9d6-b0564c3bad18 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:07.440493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 564/1118] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:07.446873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7800b4-412b-445e-a5bc-86a106c0cfcf None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:07.447586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7800b4-412b-445e-a5bc-86a106c0cfcf None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:07.448229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7800b4-412b-445e-a5bc-86a106c0cfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:07.465751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3d7800b4-412b-445e-a5bc-86a106c0cfcf None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:07.466959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3d7800b4-412b-445e-a5bc-86a106c0cfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:07.535850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 555/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:07.558346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b780ed-030f-4fd1-b9d6-b0564c3bad18 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:07.668817 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b780ed-030f-4fd1-b9d6-b0564c3bad18 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:07.685234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:07.704669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38773871-2f96-40e8-b9b2-af18d4ea9e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:07.705461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38773871-2f96-40e8-b9b2-af18d4ea9e05 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:07.705742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38773871-2f96-40e8-b9b2-af18d4ea9e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:07.706390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38773871-2f96-40e8-b9b2-af18d4ea9e05 None admin] PATH_INFO: `/v3/roles/ffab93457c0f4f64a29764f17759396e` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:07.707892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38773871-2f96-40e8-b9b2-af18d4ea9e05 None admin] RBAC: Authorizing `identity:get_role(role_id=ffab93457c0f4f64a29764f17759396e)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:07.709263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38773871-2f96-40e8-b9b2-af18d4ea9e05 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:07.710653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 565/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:07 2022] GET /identity/v3/roles/ffab93457c0f4f64a29764f17759396e => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:07.717503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7800b4-412b-445e-a5bc-86a106c0cfcf None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:07.788780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:07.843087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7800b4-412b-445e-a5bc-86a106c0cfcf None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:07.880714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7791d4-6c6e-4b4b-9f93-f99d46768534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:07.881586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7791d4-6c6e-4b4b-9f93-f99d46768534 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:07.881862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7791d4-6c6e-4b4b-9f93-f99d46768534 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:07.882204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7791d4-6c6e-4b4b-9f93-f99d46768534 None admin] PATH_INFO: `/v3/users/979650d6056c4d71b67c35d96e206589` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:07.883454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7791d4-6c6e-4b4b-9f93-f99d46768534 None admin] RBAC: Authorizing `identity:get_user(user_id=979650d6056c4d71b67c35d96e206589)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:07.884947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7791d4-6c6e-4b4b-9f93-f99d46768534 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:07.886658 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 556/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:07 2022] GET /identity/v3/users/979650d6056c4d71b67c35d96e206589 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:07.892812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38773871-2f96-40e8-b9b2-af18d4ea9e05 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:08.023120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38773871-2f96-40e8-b9b2-af18d4ea9e05 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:08.061632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e1a30-e1a2-4236-92ee-98871f90bdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:08.062508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e1a30-e1a2-4236-92ee-98871f90bdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:08.062786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e1a30-e1a2-4236-92ee-98871f90bdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:08.063055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e1a30-e1a2-4236-92ee-98871f90bdf2 None admin] PATH_INFO: `/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:08.064273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e1a30-e1a2-4236-92ee-98871f90bdf2 None admin] RBAC: Authorizing `identity:get_project(project_id=3fd4b4a1abd349b2ac625c514aff32c1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:08.065708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e1a30-e1a2-4236-92ee-98871f90bdf2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:08.067429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 566/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:53:07 2022] GET /identity/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:08.074078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7791d4-6c6e-4b4b-9f93-f99d46768534 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:08.190520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7791d4-6c6e-4b4b-9f93-f99d46768534 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:08.219554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:08.226893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ecec8f-294d-408b-b03b-a35f04d4473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:08.227785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecec8f-294d-408b-b03b-a35f04d4473f None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:08.228069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecec8f-294d-408b-b03b-a35f04d4473f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:08.228671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecec8f-294d-408b-b03b-a35f04d4473f None admin] PATH_INFO: `/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1/users/979650d6056c4d71b67c35d96e206589/roles/ffab93457c0f4f64a29764f17759396e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:08.230787 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ecec8f-294d-408b-b03b-a35f04d4473f None admin] RBAC: Authorizing `identity:create_grant(project_id=3fd4b4a1abd349b2ac625c514aff32c1, user_id=979650d6056c4d71b67c35d96e206589, role_id=ffab93457c0f4f64a29764f17759396e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:08.232247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ecec8f-294d-408b-b03b-a35f04d4473f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:08.247337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 557/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:53:08 2022] PUT /identity/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1/users/979650d6056c4d71b67c35d96e206589/roles/ffab93457c0f4f64a29764f17759396e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:53:10.216910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c460e78-54ac-4aec-a49a-840a535c7afc None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:10.217446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c460e78-54ac-4aec-a49a-840a535c7afc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:10.217446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c460e78-54ac-4aec-a49a-840a535c7afc None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:10.218821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 567/1124] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:10.225226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c2813-290d-4193-9f51-57e0dad4f867 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:10.225923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c2813-290d-4193-9f51-57e0dad4f867 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:10.226145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c2813-290d-4193-9f51-57e0dad4f867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:10.251415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-df2c2813-290d-4193-9f51-57e0dad4f867 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:10.252611 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-df2c2813-290d-4193-9f51-57e0dad4f867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:10.334417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 558/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:10.358067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c460e78-54ac-4aec-a49a-840a535c7afc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:10.420364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:10.470175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c460e78-54ac-4aec-a49a-840a535c7afc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:10.505065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a67538-0d4d-41e0-a273-f381d0e8a51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:10.505918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a67538-0d4d-41e0-a273-f381d0e8a51d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:10.506241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a67538-0d4d-41e0-a273-f381d0e8a51d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:10.506517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a67538-0d4d-41e0-a273-f381d0e8a51d None admin] PATH_INFO: `/v3/roles/ffab93457c0f4f64a29764f17759396e` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:10.508103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a67538-0d4d-41e0-a273-f381d0e8a51d None admin] RBAC: Authorizing `identity:get_role(role_id=ffab93457c0f4f64a29764f17759396e)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:10.509858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a67538-0d4d-41e0-a273-f381d0e8a51d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:10.510936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 568/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:10 2022] GET /identity/v3/roles/ffab93457c0f4f64a29764f17759396e => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:10.517056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c2813-290d-4193-9f51-57e0dad4f867 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:10.627807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c2813-290d-4193-9f51-57e0dad4f867 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:10.649567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:10.667663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bf11d-60d3-42e6-96f6-ac6e138d357c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:10.668548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf11d-60d3-42e6-96f6-ac6e138d357c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:10.668828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf11d-60d3-42e6-96f6-ac6e138d357c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:10.669104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7bf11d-60d3-42e6-96f6-ac6e138d357c None admin] PATH_INFO: `/v3/users/979650d6056c4d71b67c35d96e206589` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:10.670387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bf11d-60d3-42e6-96f6-ac6e138d357c None admin] RBAC: Authorizing `identity:get_user(user_id=979650d6056c4d71b67c35d96e206589)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:10.671848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7bf11d-60d3-42e6-96f6-ac6e138d357c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:10.673544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 559/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:10 2022] GET /identity/v3/users/979650d6056c4d71b67c35d96e206589 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:10.679427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a67538-0d4d-41e0-a273-f381d0e8a51d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:10.786242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a67538-0d4d-41e0-a273-f381d0e8a51d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:10.806935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:10.828079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375999d-1263-4503-8cc8-37085f681d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:10.828721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375999d-1263-4503-8cc8-37085f681d9e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:10.829000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375999d-1263-4503-8cc8-37085f681d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:10.829275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375999d-1263-4503-8cc8-37085f681d9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:10.907389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375999d-1263-4503-8cc8-37085f681d9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:10.908845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375999d-1263-4503-8cc8-37085f681d9e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:11.009716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 569/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:53:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:53:11.019300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bf11d-60d3-42e6-96f6-ac6e138d357c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:11.121726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7bf11d-60d3-42e6-96f6-ac6e138d357c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:11.156746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304dc8e-fb60-445d-a5d1-afaf183c3f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:11.157600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304dc8e-fb60-445d-a5d1-afaf183c3f3c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:11.157881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304dc8e-fb60-445d-a5d1-afaf183c3f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:11.158154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304dc8e-fb60-445d-a5d1-afaf183c3f3c None admin] PATH_INFO: `/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:11.159414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304dc8e-fb60-445d-a5d1-afaf183c3f3c None admin] RBAC: Authorizing `identity:get_project(project_id=3fd4b4a1abd349b2ac625c514aff32c1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:11.160866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304dc8e-fb60-445d-a5d1-afaf183c3f3c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:11.162671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 560/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:53:11 2022] GET /identity/v3/projects/3fd4b4a1abd349b2ac625c514aff32c1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:11.169281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375999d-1263-4503-8cc8-37085f681d9e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:11.191120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:11.276760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375999d-1263-4503-8cc8-37085f681d9e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:11.312046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598ab20-e535-4462-9c92-4ca05505df26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:11.312691 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598ab20-e535-4462-9c92-4ca05505df26 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:11.312969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598ab20-e535-4462-9c92-4ca05505df26 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:11.313242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598ab20-e535-4462-9c92-4ca05505df26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:11.314064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598ab20-e535-4462-9c92-4ca05505df26 None admin] RBAC: Adding query filter params (role.id=ffab93457c0f4f64a29764f17759396e, scope.project.id=3fd4b4a1abd349b2ac625c514aff32c1, user.id=979650d6056c4d71b67c35d96e206589) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:11.314404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598ab20-e535-4462-9c92-4ca05505df26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:11.315844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598ab20-e535-4462-9c92-4ca05505df26 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:11.321768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 570/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:53:11 2022] GET /identity/v3/role_assignments?user.id=979650d6056c4d71b67c35d96e206589&scope.project.id=3fd4b4a1abd349b2ac625c514aff32c1&role.id=ffab93457c0f4f64a29764f17759396e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:13.377464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dceef-d45d-4560-b7c2-0358a64849e3 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:13.378340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dceef-d45d-4560-b7c2-0358a64849e3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:13.378340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841dceef-d45d-4560-b7c2-0358a64849e3 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:13.380633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 561/1131] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:13.386961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a663-186b-456e-b258-501cd6db73b6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:13.387671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a663-186b-456e-b258-501cd6db73b6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:13.387808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a663-186b-456e-b258-501cd6db73b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:13.409432 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7d39a663-186b-456e-b258-501cd6db73b6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:13.410650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7d39a663-186b-456e-b258-501cd6db73b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:13.479894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 571/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:13.504566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841dceef-d45d-4560-b7c2-0358a64849e3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:13.569651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:13.620604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841dceef-d45d-4560-b7c2-0358a64849e3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:13.655195 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6666e-a398-4fa7-91a7-386190b4e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:13.655922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6666e-a398-4fa7-91a7-386190b4e272 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:13.656206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6666e-a398-4fa7-91a7-386190b4e272 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:13.656482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6666e-a398-4fa7-91a7-386190b4e272 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:13.657672 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6666e-a398-4fa7-91a7-386190b4e272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:13.659153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6666e-a398-4fa7-91a7-386190b4e272 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:13.660811 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 562/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:53:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:13.667502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39a663-186b-456e-b258-501cd6db73b6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:13.684808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:13.778277 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39a663-186b-456e-b258-501cd6db73b6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:13.817487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed30423-c191-4121-a1ea-88385b349df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:13.818134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed30423-c191-4121-a1ea-88385b349df4 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:13.818441 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed30423-c191-4121-a1ea-88385b349df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:13.818706 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed30423-c191-4121-a1ea-88385b349df4 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:13.819409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed30423-c191-4121-a1ea-88385b349df4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:13.820841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed30423-c191-4121-a1ea-88385b349df4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:13.856730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 572/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:53:13 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 10:53:15.921125 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88609950-cf53-4331-b808-baeef9a54c1f None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:15.922019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88609950-cf53-4331-b808-baeef9a54c1f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:15.922019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88609950-cf53-4331-b808-baeef9a54c1f None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:15.924106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 563/1135] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:15.932256 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4d121-2974-4093-b2cc-9592e68d0171 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:15.932677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4d121-2974-4093-b2cc-9592e68d0171 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:15.932789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4d121-2974-4093-b2cc-9592e68d0171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:15.951767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-64d4d121-2974-4093-b2cc-9592e68d0171 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:15.952964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-64d4d121-2974-4093-b2cc-9592e68d0171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:16.016713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:16.026666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 573/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:16.051781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88609950-cf53-4331-b808-baeef9a54c1f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:16.156208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88609950-cf53-4331-b808-baeef9a54c1f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:16.179892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:16.192608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae545e-df05-4036-bd49-1c97753b0e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:16.193391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae545e-df05-4036-bd49-1c97753b0e06 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:16.193677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae545e-df05-4036-bd49-1c97753b0e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:16.193947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae545e-df05-4036-bd49-1c97753b0e06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:16.195358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae545e-df05-4036-bd49-1c97753b0e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:16.196830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae545e-df05-4036-bd49-1c97753b0e06 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:16.198470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 564/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:53:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:16.204446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4d121-2974-4093-b2cc-9592e68d0171 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:16.304082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4d121-2974-4093-b2cc-9592e68d0171 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:16.339997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a30120-bdaa-4bf0-840b-688ea64eb917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:16.340631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a30120-bdaa-4bf0-840b-688ea64eb917 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:16.340944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a30120-bdaa-4bf0-840b-688ea64eb917 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:16.341224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a30120-bdaa-4bf0-840b-688ea64eb917 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:16.341930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a30120-bdaa-4bf0-840b-688ea64eb917 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:16.343424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a30120-bdaa-4bf0-840b-688ea64eb917 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:16.370473 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 574/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:53:16 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:53:18.447218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08cb9-d803-440c-a95c-ed69b0e41bc9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:18.448024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08cb9-d803-440c-a95c-ed69b0e41bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:18.448290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08cb9-d803-440c-a95c-ed69b0e41bc9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:18.450607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 565/1139] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:18.456156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b09599-3ab2-4823-8f52-d8b24021201e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:18.456588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b09599-3ab2-4823-8f52-d8b24021201e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:18.456691 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b09599-3ab2-4823-8f52-d8b24021201e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:18.475595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-20b09599-3ab2-4823-8f52-d8b24021201e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:18.476781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-20b09599-3ab2-4823-8f52-d8b24021201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:18.528683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:18.560687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 575/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:18.595740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c08cb9-d803-440c-a95c-ed69b0e41bc9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:18.716713 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c08cb9-d803-440c-a95c-ed69b0e41bc9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:18.750697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d5e47-f45c-42d1-ac51-5957f2602f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:18.751493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d5e47-f45c-42d1-ac51-5957f2602f4e None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:18.751816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d5e47-f45c-42d1-ac51-5957f2602f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:18.752088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d5e47-f45c-42d1-ac51-5957f2602f4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:18.765095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d5e47-f45c-42d1-ac51-5957f2602f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-8d7d5e47-f45c-42d1-ac51-5957f2602f4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:53:18.772548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:53:18.773774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:53:18.775997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 566/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:53:18.783196 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b09599-3ab2-4823-8f52-d8b24021201e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:18.891354 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b09599-3ab2-4823-8f52-d8b24021201e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:18.927701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38294711-d557-45b7-8dc2-3d11f104818f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:18.928375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38294711-d557-45b7-8dc2-3d11f104818f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:18.928655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38294711-d557-45b7-8dc2-3d11f104818f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:18.928929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38294711-d557-45b7-8dc2-3d11f104818f None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:18.929679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38294711-d557-45b7-8dc2-3d11f104818f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:18.929992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38294711-d557-45b7-8dc2-3d11f104818f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:18.931635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38294711-d557-45b7-8dc2-3d11f104818f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:18.940208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 576/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:53:18.946232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d5e47-f45c-42d1-ac51-5957f2602f4e None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:19.005616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:19.056229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d5e47-f45c-42d1-ac51-5957f2602f4e None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:19.094595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae67fa-ca63-4187-9c38-822e94d1f3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:19.095468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae67fa-ca63-4187-9c38-822e94d1f3bb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:19.095756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae67fa-ca63-4187-9c38-822e94d1f3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:19.096028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae67fa-ca63-4187-9c38-822e94d1f3bb None admin] PATH_INFO: `/v3/users/526158d6a8604bc29cc342b298da8c72` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:19.109794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae67fa-ca63-4187-9c38-822e94d1f3bb None admin] RBAC: Authorizing `identity:get_user(user_id=526158d6a8604bc29cc342b298da8c72)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:19.111295 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae67fa-ca63-4187-9c38-822e94d1f3bb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:19.112934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 567/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:18 2022] GET /identity/v3/users/526158d6a8604bc29cc342b298da8c72 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:19.119131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38294711-d557-45b7-8dc2-3d11f104818f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:19.167218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:19.224941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38294711-d557-45b7-8dc2-3d11f104818f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:19.259637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f7d5f-b2a5-4122-880d-0bf6659db0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:19.260216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f7d5f-b2a5-4122-880d-0bf6659db0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:19.260493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f7d5f-b2a5-4122-880d-0bf6659db0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:19.260770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f7d5f-b2a5-4122-880d-0bf6659db0e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:19.336308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f7d5f-b2a5-4122-880d-0bf6659db0e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:19.338007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f7d5f-b2a5-4122-880d-0bf6659db0e3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:19.382647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:19.439316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 577/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:53:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:53:19.448514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae67fa-ca63-4187-9c38-822e94d1f3bb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:19.561574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae67fa-ca63-4187-9c38-822e94d1f3bb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:19.572511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:19.601877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7af067-2f8b-45b0-9371-c8555f5a23bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:19.602752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af067-2f8b-45b0-9371-c8555f5a23bf None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:19.603224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af067-2f8b-45b0-9371-c8555f5a23bf None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:19.603506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7af067-2f8b-45b0-9371-c8555f5a23bf None admin] PATH_INFO: `/v3/projects/fafbdec72987494a9923fa5e410cb2a8` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:19.604735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7af067-2f8b-45b0-9371-c8555f5a23bf None admin] RBAC: Authorizing `identity:get_project(project_id=fafbdec72987494a9923fa5e410cb2a8)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:19.606208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7af067-2f8b-45b0-9371-c8555f5a23bf None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:19.607986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 568/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:53:19 2022] GET /identity/v3/projects/fafbdec72987494a9923fa5e410cb2a8 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:19.614164 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f7d5f-b2a5-4122-880d-0bf6659db0e3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:19.719940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f7d5f-b2a5-4122-880d-0bf6659db0e3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:19.756550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e70633-4a0c-46f9-b4dc-e3fa32c0afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:19.757198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e70633-4a0c-46f9-b4dc-e3fa32c0afbc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:19.757480 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e70633-4a0c-46f9-b4dc-e3fa32c0afbc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:19.757752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e70633-4a0c-46f9-b4dc-e3fa32c0afbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:19.758622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e70633-4a0c-46f9-b4dc-e3fa32c0afbc None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=fafbdec72987494a9923fa5e410cb2a8, user.id=526158d6a8604bc29cc342b298da8c72) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:19.758934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e70633-4a0c-46f9-b4dc-e3fa32c0afbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:19.760461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e70633-4a0c-46f9-b4dc-e3fa32c0afbc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:19.766063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 578/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:53:19 2022] GET /identity/v3/role_assignments?user.id=526158d6a8604bc29cc342b298da8c72&scope.project.id=fafbdec72987494a9923fa5e410cb2a8&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:21.761576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7f7dc-c249-4ed4-94d8-ecdc4593cfe9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:21.762184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7f7dc-c249-4ed4-94d8-ecdc4593cfe9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:21.762184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7f7dc-c249-4ed4-94d8-ecdc4593cfe9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:21.763532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 569/1147] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:21.769845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2ff51-5922-4687-a992-c49b991f0f05 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:21.770306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2ff51-5922-4687-a992-c49b991f0f05 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:21.770469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2ff51-5922-4687-a992-c49b991f0f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:21.796250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-41d2ff51-5922-4687-a992-c49b991f0f05 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:21.797446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-41d2ff51-5922-4687-a992-c49b991f0f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:21.799474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:21.871861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 579/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:21.900018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7f7dc-c249-4ed4-94d8-ecdc4593cfe9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:22.018969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7f7dc-c249-4ed4-94d8-ecdc4593cfe9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:22.060947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83430-ab04-4f2e-a183-ad0e8636f507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:22.061743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83430-ab04-4f2e-a183-ad0e8636f507 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:22.062027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83430-ab04-4f2e-a183-ad0e8636f507 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:22.062341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc83430-ab04-4f2e-a183-ad0e8636f507 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:22.074285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc83430-ab04-4f2e-a183-ad0e8636f507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-3cc83430-ab04-4f2e-a183-ad0e8636f507 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:53:22.082235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:53:22.083305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:53:22.084403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 570/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:53:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:53:22.091285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2ff51-5922-4687-a992-c49b991f0f05 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:22.194194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2ff51-5922-4687-a992-c49b991f0f05 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:22.230487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa561eca-b2a6-42b5-80d9-011d6a46ee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:22.231143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa561eca-b2a6-42b5-80d9-011d6a46ee19 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:22.231427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa561eca-b2a6-42b5-80d9-011d6a46ee19 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:22.231707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa561eca-b2a6-42b5-80d9-011d6a46ee19 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:22.232454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa561eca-b2a6-42b5-80d9-011d6a46ee19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:22.232763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa561eca-b2a6-42b5-80d9-011d6a46ee19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:22.234134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa561eca-b2a6-42b5-80d9-011d6a46ee19 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:22.239304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:22.242792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 580/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:53:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:22.249360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83430-ab04-4f2e-a183-ad0e8636f507 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:22.302023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:22.369789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc83430-ab04-4f2e-a183-ad0e8636f507 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:22.405049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99bbaa-e6c2-432e-a5e0-8b4a922be973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:22.405904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99bbaa-e6c2-432e-a5e0-8b4a922be973 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:22.406218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99bbaa-e6c2-432e-a5e0-8b4a922be973 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:22.406492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99bbaa-e6c2-432e-a5e0-8b4a922be973 None admin] PATH_INFO: `/v3/users/526158d6a8604bc29cc342b298da8c72` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:22.407776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99bbaa-e6c2-432e-a5e0-8b4a922be973 None admin] RBAC: Authorizing `identity:get_user(user_id=526158d6a8604bc29cc342b298da8c72)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:22.409222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99bbaa-e6c2-432e-a5e0-8b4a922be973 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:22.410856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 571/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:22 2022] GET /identity/v3/users/526158d6a8604bc29cc342b298da8c72 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:22.420036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa561eca-b2a6-42b5-80d9-011d6a46ee19 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:22.528076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa561eca-b2a6-42b5-80d9-011d6a46ee19 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:22.563515 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56534569-3807-4474-ad53-80d82408e978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:22.564311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56534569-3807-4474-ad53-80d82408e978 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:22.564596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56534569-3807-4474-ad53-80d82408e978 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:22.564872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56534569-3807-4474-ad53-80d82408e978 None admin] PATH_INFO: `/v3/projects/fafbdec72987494a9923fa5e410cb2a8` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:22.566211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56534569-3807-4474-ad53-80d82408e978 None admin] RBAC: Authorizing `identity:get_project(project_id=fafbdec72987494a9923fa5e410cb2a8)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:22.567657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56534569-3807-4474-ad53-80d82408e978 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:22.569379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 581/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:53:22 2022] GET /identity/v3/projects/fafbdec72987494a9923fa5e410cb2a8 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:22.576067 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99bbaa-e6c2-432e-a5e0-8b4a922be973 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:22.687254 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99bbaa-e6c2-432e-a5e0-8b4a922be973 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:22.694149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:22.723809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deef391-9118-4fe7-8878-c46e056847c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:22.724700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deef391-9118-4fe7-8878-c46e056847c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:22.724983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deef391-9118-4fe7-8878-c46e056847c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:22.725255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deef391-9118-4fe7-8878-c46e056847c6 None admin] PATH_INFO: `/v3/projects/fafbdec72987494a9923fa5e410cb2a8/users/526158d6a8604bc29cc342b298da8c72/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:22.727430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deef391-9118-4fe7-8878-c46e056847c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=fafbdec72987494a9923fa5e410cb2a8, user_id=526158d6a8604bc29cc342b298da8c72, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:22.729411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deef391-9118-4fe7-8878-c46e056847c6 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:22.748415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 572/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:53:22 2022] PUT /identity/v3/projects/fafbdec72987494a9923fa5e410cb2a8/users/526158d6a8604bc29cc342b298da8c72/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:53:24.778770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b695-7fd5-464d-9b1b-177517c3c069 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:24.779400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b695-7fd5-464d-9b1b-177517c3c069 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:24.779400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b695-7fd5-464d-9b1b-177517c3c069 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:24.780722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 582/1154] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:24.787139 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366f3a-8780-4df0-bb1e-243aecc2f4b4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:24.787889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366f3a-8780-4df0-bb1e-243aecc2f4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:24.788045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366f3a-8780-4df0-bb1e-243aecc2f4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:24.808344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5a366f3a-8780-4df0-bb1e-243aecc2f4b4 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:24.809579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5a366f3a-8780-4df0-bb1e-243aecc2f4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:24.883000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 573/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:24.906658 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864b695-7fd5-464d-9b1b-177517c3c069 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:24.975658 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:25.012269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864b695-7fd5-464d-9b1b-177517c3c069 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:25.049085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fa075-97ef-4c30-97ae-1594d0441d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:25.049811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fa075-97ef-4c30-97ae-1594d0441d37 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:25.050124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fa075-97ef-4c30-97ae-1594d0441d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:25.050401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fa075-97ef-4c30-97ae-1594d0441d37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:25.061338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fa075-97ef-4c30-97ae-1594d0441d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-038fa075-97ef-4c30-97ae-1594d0441d37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:53:25.068962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:53:25.070002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:53:25.071174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 583/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:53:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:53:25.079633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a366f3a-8780-4df0-bb1e-243aecc2f4b4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:25.183235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:25.191935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a366f3a-8780-4df0-bb1e-243aecc2f4b4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:25.236117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef5e8-a952-4309-acd2-9e7c6297aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:25.236777 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef5e8-a952-4309-acd2-9e7c6297aa3f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:25.237057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef5e8-a952-4309-acd2-9e7c6297aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:25.237331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef5e8-a952-4309-acd2-9e7c6297aa3f None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:25.238123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ef5e8-a952-4309-acd2-9e7c6297aa3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:25.238422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ef5e8-a952-4309-acd2-9e7c6297aa3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:25.239811 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ef5e8-a952-4309-acd2-9e7c6297aa3f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:25.247989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 574/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:53:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:25.257097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fa075-97ef-4c30-97ae-1594d0441d37 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:25.368064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fa075-97ef-4c30-97ae-1594d0441d37 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:25.396454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:25.405620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6943d-0961-4a4e-9d16-962d3e1e28bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:25.406524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6943d-0961-4a4e-9d16-962d3e1e28bd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:25.406806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6943d-0961-4a4e-9d16-962d3e1e28bd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:25.407081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6943d-0961-4a4e-9d16-962d3e1e28bd None admin] PATH_INFO: `/v3/users/526158d6a8604bc29cc342b298da8c72` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:25.408300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6943d-0961-4a4e-9d16-962d3e1e28bd None admin] RBAC: Authorizing `identity:get_user(user_id=526158d6a8604bc29cc342b298da8c72)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:25.409737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6943d-0961-4a4e-9d16-962d3e1e28bd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:25.411396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 584/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:25 2022] GET /identity/v3/users/526158d6a8604bc29cc342b298da8c72 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:25.417552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef5e8-a952-4309-acd2-9e7c6297aa3f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:25.522304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef5e8-a952-4309-acd2-9e7c6297aa3f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:25.557512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e92a7-f1a7-4625-90b7-c5c0911cd2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:25.558150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e92a7-f1a7-4625-90b7-c5c0911cd2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:25.558449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e92a7-f1a7-4625-90b7-c5c0911cd2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:25.558738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e92a7-f1a7-4625-90b7-c5c0911cd2d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:25.586679 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:25.631101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e92a7-f1a7-4625-90b7-c5c0911cd2d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:25.632495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e92a7-f1a7-4625-90b7-c5c0911cd2d8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:25.747005 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 575/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:53:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:53:25.753374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6943d-0961-4a4e-9d16-962d3e1e28bd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:25.862243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6943d-0961-4a4e-9d16-962d3e1e28bd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:25.901408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a0b31-6c28-47f7-9aa3-f4f0ce5431fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:25.902229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a0b31-6c28-47f7-9aa3-f4f0ce5431fa None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:25.902512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a0b31-6c28-47f7-9aa3-f4f0ce5431fa None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:25.902808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a0b31-6c28-47f7-9aa3-f4f0ce5431fa None admin] PATH_INFO: `/v3/projects/fafbdec72987494a9923fa5e410cb2a8` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:25.904110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a0b31-6c28-47f7-9aa3-f4f0ce5431fa None admin] RBAC: Authorizing `identity:get_project(project_id=fafbdec72987494a9923fa5e410cb2a8)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:25.905548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a0b31-6c28-47f7-9aa3-f4f0ce5431fa None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:25.907274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 585/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:53:25 2022] GET /identity/v3/projects/fafbdec72987494a9923fa5e410cb2a8 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:25.913646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e92a7-f1a7-4625-90b7-c5c0911cd2d8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:25.972821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:26.015697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e92a7-f1a7-4625-90b7-c5c0911cd2d8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:26.062597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea060e3-5151-4fb4-8284-c9cde857b54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:26.063250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea060e3-5151-4fb4-8284-c9cde857b54c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:26.063540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea060e3-5151-4fb4-8284-c9cde857b54c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:26.063803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea060e3-5151-4fb4-8284-c9cde857b54c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:26.064638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea060e3-5151-4fb4-8284-c9cde857b54c None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=fafbdec72987494a9923fa5e410cb2a8, user.id=526158d6a8604bc29cc342b298da8c72) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:26.064947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea060e3-5151-4fb4-8284-c9cde857b54c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:26.066412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea060e3-5151-4fb4-8284-c9cde857b54c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:26.072768 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 576/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:53:25 2022] GET /identity/v3/role_assignments?user.id=526158d6a8604bc29cc342b298da8c72&scope.project.id=fafbdec72987494a9923fa5e410cb2a8&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:28.111572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e67c-3bf4-4576-92db-e302116a969c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:28.112405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e67c-3bf4-4576-92db-e302116a969c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:28.112405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78e67c-3bf4-4576-92db-e302116a969c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:28.114708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 586/1162] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:28.122617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a3dc-a092-4125-864a-3d886f726624 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:28.123033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a3dc-a092-4125-864a-3d886f726624 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:28.123189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63a3dc-a092-4125-864a-3d886f726624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:28.144115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bc63a3dc-a092-4125-864a-3d886f726624 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:28.145295 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bc63a3dc-a092-4125-864a-3d886f726624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:28.216189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 577/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:28.239100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78e67c-3bf4-4576-92db-e302116a969c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:28.312606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:28.357153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78e67c-3bf4-4576-92db-e302116a969c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:28.394013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad11dc-5709-46d7-887c-2386d0d65047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:28.395315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad11dc-5709-46d7-887c-2386d0d65047 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:28.395574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad11dc-5709-46d7-887c-2386d0d65047 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:28.395905 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad11dc-5709-46d7-887c-2386d0d65047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:28.475332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad11dc-5709-46d7-887c-2386d0d65047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:28.476709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad11dc-5709-46d7-887c-2386d0d65047 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:28.546057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:28.586540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 587/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:53:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:53:28.593260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63a3dc-a092-4125-864a-3d886f726624 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:28.651988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:28.708035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63a3dc-a092-4125-864a-3d886f726624 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:28.745317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4725a5-ece7-497f-8d8f-9d9ffb3dbdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:28.746047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4725a5-ece7-497f-8d8f-9d9ffb3dbdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:28.746356 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4725a5-ece7-497f-8d8f-9d9ffb3dbdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:28.746630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4725a5-ece7-497f-8d8f-9d9ffb3dbdf5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:28.753646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4725a5-ece7-497f-8d8f-9d9ffb3dbdf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:28.755145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4725a5-ece7-497f-8d8f-9d9ffb3dbdf5 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-be4725a5-ece7-497f-8d8f-9d9ffb3dbdf5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:53:28.769499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:53:28.770668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:53:28.771533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 578/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jun 20 10:53:28 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:53:28.779448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad11dc-5709-46d7-887c-2386d0d65047 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:28.890419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad11dc-5709-46d7-887c-2386d0d65047 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:28.934940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc9652-8fc9-48ba-b328-bf7434118aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:28.935613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc9652-8fc9-48ba-b328-bf7434118aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:28.935899 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc9652-8fc9-48ba-b328-bf7434118aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:28.936173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc9652-8fc9-48ba-b328-bf7434118aa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:28.936898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc9652-8fc9-48ba-b328-bf7434118aa8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:28.937213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc9652-8fc9-48ba-b328-bf7434118aa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:28.938765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc9652-8fc9-48ba-b328-bf7434118aa8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:28.946146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 588/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Jun 20 10:53:28 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:53:30.954293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529078c-d6ad-4e9f-ba0a-e4bfeaa5956c None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:30.955124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529078c-d6ad-4e9f-ba0a-e4bfeaa5956c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:30.955643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529078c-d6ad-4e9f-ba0a-e4bfeaa5956c None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:30.957816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 579/1167] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:30.963312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7748da8-908e-4a1f-a03d-fce52a4151ca None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:30.963746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7748da8-908e-4a1f-a03d-fce52a4151ca None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:30.963837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7748da8-908e-4a1f-a03d-fce52a4151ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:30.982564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b7748da8-908e-4a1f-a03d-fce52a4151ca None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:30.983842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b7748da8-908e-4a1f-a03d-fce52a4151ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:31.007151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:31.070847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 589/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:31.094894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529078c-d6ad-4e9f-ba0a-e4bfeaa5956c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:31.214649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529078c-d6ad-4e9f-ba0a-e4bfeaa5956c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:31.227500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:31.252952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03225f6-2281-4f29-94da-34c6d1066fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:31.253614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03225f6-2281-4f29-94da-34c6d1066fd9 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:31.253896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03225f6-2281-4f29-94da-34c6d1066fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:31.254210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03225f6-2281-4f29-94da-34c6d1066fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:31.254957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03225f6-2281-4f29-94da-34c6d1066fd9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:31.256527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03225f6-2281-4f29-94da-34c6d1066fd9 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:31.273420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 580/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Mon Jun 20 10:53:31 2022] POST /identity/v3/domains => generated 267 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:53:33.323947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418e90f-9797-46bc-86e9-623cfb757ac8 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:33.324552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418e90f-9797-46bc-86e9-623cfb757ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:33.324552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418e90f-9797-46bc-86e9-623cfb757ac8 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:33.325912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 590/1170] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:33.332799 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d6649-8b12-4250-b332-c93cbe4f7ef7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:33.333542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d6649-8b12-4250-b332-c93cbe4f7ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:33.333706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d6649-8b12-4250-b332-c93cbe4f7ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:33.352883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-796d6649-8b12-4250-b332-c93cbe4f7ef7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:33.354069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-796d6649-8b12-4250-b332-c93cbe4f7ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:33.424947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 581/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:33.451403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418e90f-9797-46bc-86e9-623cfb757ac8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:33.565861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418e90f-9797-46bc-86e9-623cfb757ac8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:33.601519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4c255-a88b-4607-bb42-afa58f6d2158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:33.602339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4c255-a88b-4607-bb42-afa58f6d2158 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:33.602623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4c255-a88b-4607-bb42-afa58f6d2158 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:33.602922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4c255-a88b-4607-bb42-afa58f6d2158 None admin] PATH_INFO: `/v3/domains/ee75be5b0bcb4451a706f393ec1f1774` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:33.613677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4c255-a88b-4607-bb42-afa58f6d2158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ee75be5b0bcb4451a706f393ec1f1774)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:33.615166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4c255-a88b-4607-bb42-afa58f6d2158 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:33.616804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 591/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Jun 20 10:53:33 2022] GET /identity/v3/domains/ee75be5b0bcb4451a706f393ec1f1774 => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:33.623606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d6649-8b12-4250-b332-c93cbe4f7ef7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:33.737362 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d6649-8b12-4250-b332-c93cbe4f7ef7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:33.738977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:33.775816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672aeda-be63-40a7-b7fd-26db78bb7d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:33.776466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672aeda-be63-40a7-b7fd-26db78bb7d4a None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:33.776744 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672aeda-be63-40a7-b7fd-26db78bb7d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:33.777013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672aeda-be63-40a7-b7fd-26db78bb7d4a None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:33.777716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672aeda-be63-40a7-b7fd-26db78bb7d4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:33.779197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672aeda-be63-40a7-b7fd-26db78bb7d4a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:33.806875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 582/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:53:33 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:53:35.835150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f93a1-f813-46d2-9fb2-564f12dd40f2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:35.835698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f93a1-f813-46d2-9fb2-564f12dd40f2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:35.835698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f93a1-f813-46d2-9fb2-564f12dd40f2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:35.838216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 592/1174] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:35.843981 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa8c10-69ec-40c3-bf35-1f926ee149ec None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:35.844445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa8c10-69ec-40c3-bf35-1f926ee149ec None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:35.844544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa8c10-69ec-40c3-bf35-1f926ee149ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:35.861624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-35fa8c10-69ec-40c3-bf35-1f926ee149ec None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:35.862828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-35fa8c10-69ec-40c3-bf35-1f926ee149ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:35.945644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 583/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:53:35.970972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f93a1-f813-46d2-9fb2-564f12dd40f2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:35.994479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:36.085546 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f93a1-f813-46d2-9fb2-564f12dd40f2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:36.120153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc893c5-ce54-4b91-ac94-e772fe86e5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:36.121020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc893c5-ce54-4b91-ac94-e772fe86e5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:36.121301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc893c5-ce54-4b91-ac94-e772fe86e5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:36.121572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc893c5-ce54-4b91-ac94-e772fe86e5d0 None admin] PATH_INFO: `/v3/domains/ee75be5b0bcb4451a706f393ec1f1774` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:36.122771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc893c5-ce54-4b91-ac94-e772fe86e5d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ee75be5b0bcb4451a706f393ec1f1774)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:36.124218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc893c5-ce54-4b91-ac94-e772fe86e5d0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:36.125831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 593/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Jun 20 10:53:35 2022] GET /identity/v3/domains/ee75be5b0bcb4451a706f393ec1f1774 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:36.132310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa8c10-69ec-40c3-bf35-1f926ee149ec None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:36.229591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:36.247567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa8c10-69ec-40c3-bf35-1f926ee149ec None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:36.283084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6fc7ec-5c28-4903-98d8-e8ec270468c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:36.283750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fc7ec-5c28-4903-98d8-e8ec270468c2 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:36.284034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fc7ec-5c28-4903-98d8-e8ec270468c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:36.284318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6fc7ec-5c28-4903-98d8-e8ec270468c2 None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:36.285028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6fc7ec-5c28-4903-98d8-e8ec270468c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:36.286513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6fc7ec-5c28-4903-98d8-e8ec270468c2 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:36.308660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 584/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:53:36 2022] POST /identity/v3/users => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:53:38.397112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45afe9-7ad0-4054-ba77-141efebb45b2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:38.397695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45afe9-7ad0-4054-ba77-141efebb45b2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:38.397695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45afe9-7ad0-4054-ba77-141efebb45b2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:38.399093 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 594/1178] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:38.405941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6017b-4f74-43f0-9087-4478158d6e16 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:38.406739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6017b-4f74-43f0-9087-4478158d6e16 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:38.406881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6017b-4f74-43f0-9087-4478158d6e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:38.433600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-92c6017b-4f74-43f0-9087-4478158d6e16 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:38.435588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-92c6017b-4f74-43f0-9087-4478158d6e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:38.526381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 585/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:38.556404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45afe9-7ad0-4054-ba77-141efebb45b2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:38.654316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:38.672865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45afe9-7ad0-4054-ba77-141efebb45b2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:38.711888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54b441-f831-4c14-bb85-aa5ed364ac6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:38.712768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54b441-f831-4c14-bb85-aa5ed364ac6e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:38.713045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54b441-f831-4c14-bb85-aa5ed364ac6e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:38.713329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54b441-f831-4c14-bb85-aa5ed364ac6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:38.723770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54b441-f831-4c14-bb85-aa5ed364ac6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-fe54b441-f831-4c14-bb85-aa5ed364ac6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:53:38.731383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:53:38.732526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:53:38.733669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 595/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:53:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:53:38.738904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6017b-4f74-43f0-9087-4478158d6e16 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:38.809805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:38.844056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6017b-4f74-43f0-9087-4478158d6e16 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:38.880413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbcaba-b5e5-42dd-baf2-102afaf996ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:38.881061 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbcaba-b5e5-42dd-baf2-102afaf996ec None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:38.881342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbcaba-b5e5-42dd-baf2-102afaf996ec None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:38.881614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbcaba-b5e5-42dd-baf2-102afaf996ec None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:38.882388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbcaba-b5e5-42dd-baf2-102afaf996ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:38.882690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbcaba-b5e5-42dd-baf2-102afaf996ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:38.884208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbcaba-b5e5-42dd-baf2-102afaf996ec None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:38.892240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 586/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:53:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:38.898067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54b441-f831-4c14-bb85-aa5ed364ac6e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:39.004587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54b441-f831-4c14-bb85-aa5ed364ac6e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:39.044183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febb0fb-144d-4d2e-b538-3eccc54cfbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:39.045049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febb0fb-144d-4d2e-b538-3eccc54cfbfe None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:39.045329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febb0fb-144d-4d2e-b538-3eccc54cfbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:39.045596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febb0fb-144d-4d2e-b538-3eccc54cfbfe None admin] PATH_INFO: `/v3/users/faed903b25c04e0880c8f1563e68ecd0` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:39.060918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febb0fb-144d-4d2e-b538-3eccc54cfbfe None admin] RBAC: Authorizing `identity:get_user(user_id=faed903b25c04e0880c8f1563e68ecd0)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:39.062385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febb0fb-144d-4d2e-b538-3eccc54cfbfe None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:39.064005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 596/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:38 2022] GET /identity/v3/users/faed903b25c04e0880c8f1563e68ecd0 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:39.070076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbcaba-b5e5-42dd-baf2-102afaf996ec None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:39.180451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbcaba-b5e5-42dd-baf2-102afaf996ec None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:39.213287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:39.218809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb15c8-9915-4a04-87e4-65b934b13ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:39.219459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb15c8-9915-4a04-87e4-65b934b13ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:39.219739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb15c8-9915-4a04-87e4-65b934b13ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:39.220012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb15c8-9915-4a04-87e4-65b934b13ac8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:39.299580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb15c8-9915-4a04-87e4-65b934b13ac8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:39.301051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb15c8-9915-4a04-87e4-65b934b13ac8 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:39.413987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 587/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:53:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:53:39.423228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febb0fb-144d-4d2e-b538-3eccc54cfbfe None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:39.427338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:39.528590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febb0fb-144d-4d2e-b538-3eccc54cfbfe None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:39.563757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03932e3-4547-4f1e-9e86-3de315a754ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:39.564609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03932e3-4547-4f1e-9e86-3de315a754ee None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:39.564889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03932e3-4547-4f1e-9e86-3de315a754ee None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:39.565161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03932e3-4547-4f1e-9e86-3de315a754ee None admin] PATH_INFO: `/v3/projects/77f0446234594e24a3f497c433663031` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:39.566474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03932e3-4547-4f1e-9e86-3de315a754ee None admin] RBAC: Authorizing `identity:get_project(project_id=77f0446234594e24a3f497c433663031)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:39.567925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03932e3-4547-4f1e-9e86-3de315a754ee None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:39.569608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 597/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:53:39 2022] GET /identity/v3/projects/77f0446234594e24a3f497c433663031 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:39.576444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb15c8-9915-4a04-87e4-65b934b13ac8 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:39.614610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:39.685374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb15c8-9915-4a04-87e4-65b934b13ac8 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:39.721563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a89eb8-d94b-442f-913c-a80c5b991cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:39.722233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a89eb8-d94b-442f-913c-a80c5b991cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:39.722523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a89eb8-d94b-442f-913c-a80c5b991cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:39.722795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a89eb8-d94b-442f-913c-a80c5b991cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:39.723628 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a89eb8-d94b-442f-913c-a80c5b991cc1 None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=77f0446234594e24a3f497c433663031, user.id=faed903b25c04e0880c8f1563e68ecd0) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:39.723945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a89eb8-d94b-442f-913c-a80c5b991cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:39.725385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a89eb8-d94b-442f-913c-a80c5b991cc1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:39.731287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 588/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:53:39 2022] GET /identity/v3/role_assignments?user.id=faed903b25c04e0880c8f1563e68ecd0&scope.project.id=77f0446234594e24a3f497c433663031&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:41.718485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3811b-4319-4c46-af41-665fed68619b None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:41.719078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3811b-4319-4c46-af41-665fed68619b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:41.719078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3811b-4319-4c46-af41-665fed68619b None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:41.720445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 598/1186] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:41.726824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e62b3-1789-44b0-90ce-9ca2a934609d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:41.727533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e62b3-1789-44b0-90ce-9ca2a934609d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:41.728138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e62b3-1789-44b0-90ce-9ca2a934609d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:41.751802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3b8e62b3-1789-44b0-90ce-9ca2a934609d None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:41.752976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3b8e62b3-1789-44b0-90ce-9ca2a934609d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:41.831309 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 589/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:41.854940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3811b-4319-4c46-af41-665fed68619b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:41.930936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:41.962244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3811b-4319-4c46-af41-665fed68619b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:41.997484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3889440-89bf-4092-8ad1-47ef423d28d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:41.998236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3889440-89bf-4092-8ad1-47ef423d28d9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:41.998513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3889440-89bf-4092-8ad1-47ef423d28d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:41.998784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3889440-89bf-4092-8ad1-47ef423d28d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:42.010122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3889440-89bf-4092-8ad1-47ef423d28d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a3889440-89bf-4092-8ad1-47ef423d28d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:53:42.017573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:53:42.018669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:53:42.019592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 599/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:53:42.027943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e62b3-1789-44b0-90ce-9ca2a934609d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:42.059832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:42.135994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e62b3-1789-44b0-90ce-9ca2a934609d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:42.171346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004e59d-b9f3-461b-87cf-59ab649412b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:42.172005 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004e59d-b9f3-461b-87cf-59ab649412b1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:42.172283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004e59d-b9f3-461b-87cf-59ab649412b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:42.172557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004e59d-b9f3-461b-87cf-59ab649412b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:42.173302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004e59d-b9f3-461b-87cf-59ab649412b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:42.173623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004e59d-b9f3-461b-87cf-59ab649412b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:42.175002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004e59d-b9f3-461b-87cf-59ab649412b1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:42.183211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 590/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:53:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:42.189271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3889440-89bf-4092-8ad1-47ef423d28d9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:42.290815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3889440-89bf-4092-8ad1-47ef423d28d9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:42.319788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:42.325763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ceeb0-ee60-4bc5-a7d0-c4384249a0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:42.326659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ceeb0-ee60-4bc5-a7d0-c4384249a0ed None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:42.326936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ceeb0-ee60-4bc5-a7d0-c4384249a0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:42.327209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ceeb0-ee60-4bc5-a7d0-c4384249a0ed None admin] PATH_INFO: `/v3/users/faed903b25c04e0880c8f1563e68ecd0` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:42.328427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ceeb0-ee60-4bc5-a7d0-c4384249a0ed None admin] RBAC: Authorizing `identity:get_user(user_id=faed903b25c04e0880c8f1563e68ecd0)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:42.329870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ceeb0-ee60-4bc5-a7d0-c4384249a0ed None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:42.331886 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 600/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:42 2022] GET /identity/v3/users/faed903b25c04e0880c8f1563e68ecd0 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:53:42.337955 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004e59d-b9f3-461b-87cf-59ab649412b1 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:42.433549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:42.446438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004e59d-b9f3-461b-87cf-59ab649412b1 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:42.486282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df28d6-706f-456d-a81c-6c79945fd4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:42.487145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df28d6-706f-456d-a81c-6c79945fd4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:42.487428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df28d6-706f-456d-a81c-6c79945fd4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:42.487699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df28d6-706f-456d-a81c-6c79945fd4d3 None admin] PATH_INFO: `/v3/projects/77f0446234594e24a3f497c433663031` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:42.489008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df28d6-706f-456d-a81c-6c79945fd4d3 None admin] RBAC: Authorizing `identity:get_project(project_id=77f0446234594e24a3f497c433663031)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:42.490487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df28d6-706f-456d-a81c-6c79945fd4d3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:42.492166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 591/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:53:42 2022] GET /identity/v3/projects/77f0446234594e24a3f497c433663031 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:42.502017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ceeb0-ee60-4bc5-a7d0-c4384249a0ed None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:42.614512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ceeb0-ee60-4bc5-a7d0-c4384249a0ed None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:42.652700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289cd8c-7a26-4018-8bb0-408455b5eb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:42.653581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289cd8c-7a26-4018-8bb0-408455b5eb5f None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:42.653857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289cd8c-7a26-4018-8bb0-408455b5eb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:42.654116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289cd8c-7a26-4018-8bb0-408455b5eb5f None admin] PATH_INFO: `/v3/projects/77f0446234594e24a3f497c433663031/users/faed903b25c04e0880c8f1563e68ecd0/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:42.656062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289cd8c-7a26-4018-8bb0-408455b5eb5f None admin] RBAC: Authorizing `identity:create_grant(project_id=77f0446234594e24a3f497c433663031, user_id=faed903b25c04e0880c8f1563e68ecd0, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:42.657506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289cd8c-7a26-4018-8bb0-408455b5eb5f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:42.673629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 601/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:53:42 2022] PUT /identity/v3/projects/77f0446234594e24a3f497c433663031/users/faed903b25c04e0880c8f1563e68ecd0/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:53:44.676832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d7a4a-9cc5-417b-9d2f-c354af3a2a40 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:44.677711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d7a4a-9cc5-417b-9d2f-c354af3a2a40 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:44.677711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d7a4a-9cc5-417b-9d2f-c354af3a2a40 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:44.679915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 592/1193] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:53:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:53:44.687161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca63f6-6524-4698-9938-c7a7d50e202e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:44.687595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca63f6-6524-4698-9938-c7a7d50e202e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:44.687761 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca63f6-6524-4698-9938-c7a7d50e202e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:44.714980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-82ca63f6-6524-4698-9938-c7a7d50e202e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:53:44.716180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-82ca63f6-6524-4698-9938-c7a7d50e202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:53:44.773374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:44.785704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 602/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:53:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:53:44.809359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d7a4a-9cc5-417b-9d2f-c354af3a2a40 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:44.928278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d7a4a-9cc5-417b-9d2f-c354af3a2a40 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:44.963910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081865d7-9c7a-442f-8ec9-92a666863aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:44.964700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081865d7-9c7a-442f-8ec9-92a666863aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:44.964984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081865d7-9c7a-442f-8ec9-92a666863aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:44.965264 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081865d7-9c7a-442f-8ec9-92a666863aa0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:44.977909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081865d7-9c7a-442f-8ec9-92a666863aa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-081865d7-9c7a-442f-8ec9-92a666863aa0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:53:44.985475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:53:44.986594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:53:44.987810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 593/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:53:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:53:44.994771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca63f6-6524-4698-9938-c7a7d50e202e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:45.102444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca63f6-6524-4698-9938-c7a7d50e202e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:45.138944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358b2a3-5f19-4945-a850-e64c375e34ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:45.139599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358b2a3-5f19-4945-a850-e64c375e34ab None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:45.139873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358b2a3-5f19-4945-a850-e64c375e34ab None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:45.140143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358b2a3-5f19-4945-a850-e64c375e34ab None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:45.140885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358b2a3-5f19-4945-a850-e64c375e34ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:45.141193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358b2a3-5f19-4945-a850-e64c375e34ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:45.142591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358b2a3-5f19-4945-a850-e64c375e34ab None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:45.151223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 603/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:53:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:45.160340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081865d7-9c7a-442f-8ec9-92a666863aa0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:45.181910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:45.277791 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081865d7-9c7a-442f-8ec9-92a666863aa0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:45.314012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4be1d-7a93-414b-b731-bc6c93f9312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:45.314914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4be1d-7a93-414b-b731-bc6c93f9312c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:45.315195 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4be1d-7a93-414b-b731-bc6c93f9312c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:45.315472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4be1d-7a93-414b-b731-bc6c93f9312c None admin] PATH_INFO: `/v3/users/faed903b25c04e0880c8f1563e68ecd0` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:45.316784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4be1d-7a93-414b-b731-bc6c93f9312c None admin] RBAC: Authorizing `identity:get_user(user_id=faed903b25c04e0880c8f1563e68ecd0)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:45.318263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4be1d-7a93-414b-b731-bc6c93f9312c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:45.319891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 594/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Jun 20 10:53:45 2022] GET /identity/v3/users/faed903b25c04e0880c8f1563e68ecd0 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:45.326050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358b2a3-5f19-4945-a850-e64c375e34ab None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:45.380207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:45.443358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358b2a3-5f19-4945-a850-e64c375e34ab None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:45.479216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f39ac7-5987-4b83-9de0-0b758eadfc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:45.479851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f39ac7-5987-4b83-9de0-0b758eadfc6e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:45.480135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f39ac7-5987-4b83-9de0-0b758eadfc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:45.480402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f39ac7-5987-4b83-9de0-0b758eadfc6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:45.550488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f39ac7-5987-4b83-9de0-0b758eadfc6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:45.551944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f39ac7-5987-4b83-9de0-0b758eadfc6e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:45.607472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:45.660557 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 604/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:53:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 10:53:45.667397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4be1d-7a93-414b-b731-bc6c93f9312c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:45.766820 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:45.784522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4be1d-7a93-414b-b731-bc6c93f9312c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:45.819486 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7e79f-0288-48bf-b715-8fb28ea3ee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:45.820356 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7e79f-0288-48bf-b715-8fb28ea3ee76 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:45.820639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7e79f-0288-48bf-b715-8fb28ea3ee76 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:45.820913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7e79f-0288-48bf-b715-8fb28ea3ee76 None admin] PATH_INFO: `/v3/projects/77f0446234594e24a3f497c433663031` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:45.822363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7e79f-0288-48bf-b715-8fb28ea3ee76 None admin] RBAC: Authorizing `identity:get_project(project_id=77f0446234594e24a3f497c433663031)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:45.823814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7e79f-0288-48bf-b715-8fb28ea3ee76 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:45.825519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 595/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Jun 20 10:53:45 2022] GET /identity/v3/projects/77f0446234594e24a3f497c433663031 => generated 361 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:45.835627 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f39ac7-5987-4b83-9de0-0b758eadfc6e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:53:45.956460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f39ac7-5987-4b83-9de0-0b758eadfc6e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:53:45.993031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa32deb-f2e2-411c-9727-cdd1953717a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:53:45.995538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa32deb-f2e2-411c-9727-cdd1953717a1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:53:45.995538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa32deb-f2e2-411c-9727-cdd1953717a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:53:45.995538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa32deb-f2e2-411c-9727-cdd1953717a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:53:45.995538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa32deb-f2e2-411c-9727-cdd1953717a1 None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=77f0446234594e24a3f497c433663031, user.id=faed903b25c04e0880c8f1563e68ecd0) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:53:45.995538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa32deb-f2e2-411c-9727-cdd1953717a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:53:45.996918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa32deb-f2e2-411c-9727-cdd1953717a1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:53:46.003145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 605/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:53:45 2022] GET /identity/v3/role_assignments?user.id=faed903b25c04e0880c8f1563e68ecd0&scope.project.id=77f0446234594e24a3f497c433663031&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:53:55.814835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=24 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:53:56.000433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:41.694935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b93e91-8273-431c-9708-21e363b1886e None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:41.695543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b93e91-8273-431c-9708-21e363b1886e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:41.695543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b93e91-8273-431c-9708-21e363b1886e None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:41.697330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 596/1201] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:54:41.703062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d694cc8-68b2-453e-8f35-9aebf8d33141 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:41.703533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d694cc8-68b2-453e-8f35-9aebf8d33141 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:41.703618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d694cc8-68b2-453e-8f35-9aebf8d33141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:41.722974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2d694cc8-68b2-453e-8f35-9aebf8d33141 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:54:41.724242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2d694cc8-68b2-453e-8f35-9aebf8d33141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:54:41.799670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 606/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:54:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:54:41.828543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b93e91-8273-431c-9708-21e363b1886e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:41.954313 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b93e91-8273-431c-9708-21e363b1886e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:41.995487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88542d1a-6d88-4fe6-bb4e-bff6b98e7ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:41.996587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88542d1a-6d88-4fe6-bb4e-bff6b98e7ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:41.997039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88542d1a-6d88-4fe6-bb4e-bff6b98e7ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:41.997466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88542d1a-6d88-4fe6-bb4e-bff6b98e7ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:42.009173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88542d1a-6d88-4fe6-bb4e-bff6b98e7ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:42.011134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88542d1a-6d88-4fe6-bb4e-bff6b98e7ce3 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-88542d1a-6d88-4fe6-bb4e-bff6b98e7ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:54:42.029988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:42.031239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:42.032374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:54:42.032374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:54:42.032374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:42.032374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:54:42.033999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 597/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:42.040638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d694cc8-68b2-453e-8f35-9aebf8d33141 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:42.154639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d694cc8-68b2-453e-8f35-9aebf8d33141 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:42.185109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:42.193000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fac604-9462-4aa2-82de-e6636b1a8d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:42.194071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fac604-9462-4aa2-82de-e6636b1a8d74 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:42.194605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fac604-9462-4aa2-82de-e6636b1a8d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:42.195112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fac604-9462-4aa2-82de-e6636b1a8d74 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:42.197117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fac604-9462-4aa2-82de-e6636b1a8d74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:42.197803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fac604-9462-4aa2-82de-e6636b1a8d74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:42.199276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fac604-9462-4aa2-82de-e6636b1a8d74 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:42.219441 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 607/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:42.228244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88542d1a-6d88-4fe6-bb4e-bff6b98e7ce3 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:42.294595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:42.360074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88542d1a-6d88-4fe6-bb4e-bff6b98e7ce3 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:42.408804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54787f2a-249a-4eb3-aac7-8e4a41d01aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:42.410189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54787f2a-249a-4eb3-aac7-8e4a41d01aad None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:42.411087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54787f2a-249a-4eb3-aac7-8e4a41d01aad None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:42.411596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54787f2a-249a-4eb3-aac7-8e4a41d01aad None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:42.412543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54787f2a-249a-4eb3-aac7-8e4a41d01aad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:42.414748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54787f2a-249a-4eb3-aac7-8e4a41d01aad None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:42.450344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 598/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Jun 20 10:54:42 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:54:44.648424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3997c-2b78-4161-b86e-78a4bdb3d738 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:44.649061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3997c-2b78-4161-b86e-78a4bdb3d738 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:44.649061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3997c-2b78-4161-b86e-78a4bdb3d738 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:44.650652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 608/1206] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:54:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:54:44.656491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a957c1-9458-4a84-bcce-23e52e1bc1d5 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:44.656995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a957c1-9458-4a84-bcce-23e52e1bc1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:44.657082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a957c1-9458-4a84-bcce-23e52e1bc1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:44.681754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-16a957c1-9458-4a84-bcce-23e52e1bc1d5 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:54:44.683040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-16a957c1-9458-4a84-bcce-23e52e1bc1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:54:44.759372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 599/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:54:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:54:44.783243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3997c-2b78-4161-b86e-78a4bdb3d738 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:44.902410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3997c-2b78-4161-b86e-78a4bdb3d738 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:44.941135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c8721-951a-4a07-b945-db37f1bdfa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:44.941931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c8721-951a-4a07-b945-db37f1bdfa1e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:44.942246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c8721-951a-4a07-b945-db37f1bdfa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:44.942536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c8721-951a-4a07-b945-db37f1bdfa1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:44.955526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c8721-951a-4a07-b945-db37f1bdfa1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-198c8721-951a-4a07-b945-db37f1bdfa1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:54:44.964264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:44.965340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:54:44.966247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 609/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:54:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:44.971449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a957c1-9458-4a84-bcce-23e52e1bc1d5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:45.015589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:45.080055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a957c1-9458-4a84-bcce-23e52e1bc1d5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:45.121548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efb556-5a33-4400-bb52-82b880543964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:45.122290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efb556-5a33-4400-bb52-82b880543964 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:45.122574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efb556-5a33-4400-bb52-82b880543964 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:45.122847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efb556-5a33-4400-bb52-82b880543964 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:45.123644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efb556-5a33-4400-bb52-82b880543964 None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:45.123947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efb556-5a33-4400-bb52-82b880543964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:45.125342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efb556-5a33-4400-bb52-82b880543964 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:45.138136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 600/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:54:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:45.142340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c8721-951a-4a07-b945-db37f1bdfa1e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:45.197321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:45.251233 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c8721-951a-4a07-b945-db37f1bdfa1e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:45.299583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f729d-2b94-42ac-ac9a-050f10b3605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:45.300508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f729d-2b94-42ac-ac9a-050f10b3605c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:45.300816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f729d-2b94-42ac-ac9a-050f10b3605c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:45.301094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f729d-2b94-42ac-ac9a-050f10b3605c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:45.314274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f729d-2b94-42ac-ac9a-050f10b3605c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:45.315898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f729d-2b94-42ac-ac9a-050f10b3605c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-221f729d-2b94-42ac-ac9a-050f10b3605c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:54:45.335216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:45.336407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:45.337400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:54:45.337400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:54:45.337400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:45.337400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:54:45.337400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 610/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:45.342841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efb556-5a33-4400-bb52-82b880543964 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:45.449843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:45.453157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efb556-5a33-4400-bb52-82b880543964 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:45.489365 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13fe1a-ecd9-48fc-a88e-005d578a3477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:45.491496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13fe1a-ecd9-48fc-a88e-005d578a3477 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:45.491496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13fe1a-ecd9-48fc-a88e-005d578a3477 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:45.491496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13fe1a-ecd9-48fc-a88e-005d578a3477 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:45.491751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13fe1a-ecd9-48fc-a88e-005d578a3477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:45.491911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13fe1a-ecd9-48fc-a88e-005d578a3477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:45.493485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13fe1a-ecd9-48fc-a88e-005d578a3477 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:45.506494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 601/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:45.512654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f729d-2b94-42ac-ac9a-050f10b3605c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:45.621739 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f729d-2b94-42ac-ac9a-050f10b3605c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:45.623685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:45.665566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9061a9e-f1bd-41c1-a8ec-a636bcc7e17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:45.666751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9061a9e-f1bd-41c1-a8ec-a636bcc7e17b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:45.667144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9061a9e-f1bd-41c1-a8ec-a636bcc7e17b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:45.667578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9061a9e-f1bd-41c1-a8ec-a636bcc7e17b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:45.677143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9061a9e-f1bd-41c1-a8ec-a636bcc7e17b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:45.678850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9061a9e-f1bd-41c1-a8ec-a636bcc7e17b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-f9061a9e-f1bd-41c1-a8ec-a636bcc7e17b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:54:45.695040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:45.696421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:45.697641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:45.697641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:45.697641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:54:45.697641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:54:45.697641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:54:45.697641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:54:45.697641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:54:45.697641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:54:45.697641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:54:45.697641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:54:45.703303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 611/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:45.705425 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13fe1a-ecd9-48fc-a88e-005d578a3477 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:45.821593 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13fe1a-ecd9-48fc-a88e-005d578a3477 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:45.859839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ac14c-e400-423b-8ea9-1b221696bffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:45.860523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ac14c-e400-423b-8ea9-1b221696bffe None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:45.860818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ac14c-e400-423b-8ea9-1b221696bffe None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:45.861096 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ac14c-e400-423b-8ea9-1b221696bffe None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:45.861929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ac14c-e400-423b-8ea9-1b221696bffe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:45.862288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ac14c-e400-423b-8ea9-1b221696bffe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:45.863730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ac14c-e400-423b-8ea9-1b221696bffe None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:45.882488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 602/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jun 20 10:54:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:45.888589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9061a9e-f1bd-41c1-a8ec-a636bcc7e17b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:46.016513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9061a9e-f1bd-41c1-a8ec-a636bcc7e17b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:46.053368 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285a026-336c-4f02-a69b-b653919a223c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:46.054113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285a026-336c-4f02-a69b-b653919a223c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:46.054396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285a026-336c-4f02-a69b-b653919a223c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:46.054703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285a026-336c-4f02-a69b-b653919a223c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:46.090802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:46.131866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285a026-336c-4f02-a69b-b653919a223c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:46.133638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285a026-336c-4f02-a69b-b653919a223c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:46.245743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 612/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:54:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 10:54:46.247947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ac14c-e400-423b-8ea9-1b221696bffe None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:46.270804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:46.363287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ac14c-e400-423b-8ea9-1b221696bffe None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:46.398691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6342e-f72a-4e82-b845-d21faa076a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:46.399685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6342e-f72a-4e82-b845-d21faa076a4d None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:46.400142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6342e-f72a-4e82-b845-d21faa076a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:46.400364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6342e-f72a-4e82-b845-d21faa076a4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:46.411118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6342e-f72a-4e82-b845-d21faa076a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:46.412574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6342e-f72a-4e82-b845-d21faa076a4d None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-bea6342e-f72a-4e82-b845-d21faa076a4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:54:46.432376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:46.434009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:46.435222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:54:46.435222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:54:46.435222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:46.435222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:54:46.435222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 603/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:46.439755 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285a026-336c-4f02-a69b-b653919a223c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:46.510930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:46.557063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285a026-336c-4f02-a69b-b653919a223c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:46.594652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c55e5-78ab-4f16-b1d5-03a85f807133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:46.595329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c55e5-78ab-4f16-b1d5-03a85f807133 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:46.595611 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c55e5-78ab-4f16-b1d5-03a85f807133 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:46.595887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c55e5-78ab-4f16-b1d5-03a85f807133 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:46.596619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c55e5-78ab-4f16-b1d5-03a85f807133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:46.596924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c55e5-78ab-4f16-b1d5-03a85f807133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:46.599181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c55e5-78ab-4f16-b1d5-03a85f807133 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:46.611461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 613/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:46.617302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6342e-f72a-4e82-b845-d21faa076a4d None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:46.700031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:46.733985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6342e-f72a-4e82-b845-d21faa076a4d None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:46.772946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e816b9-8270-4dc1-96bd-ca0a8f43c08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:46.773756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e816b9-8270-4dc1-96bd-ca0a8f43c08c None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:46.774108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e816b9-8270-4dc1-96bd-ca0a8f43c08c None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:46.774776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e816b9-8270-4dc1-96bd-ca0a8f43c08c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:46.782651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e816b9-8270-4dc1-96bd-ca0a8f43c08c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:46.784276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e816b9-8270-4dc1-96bd-ca0a8f43c08c None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-21e816b9-8270-4dc1-96bd-ca0a8f43c08c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:54:46.801069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:46.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:46.808775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:54:46.808775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:54:46.808775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:54:46.808775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:54:46.808775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:54:46.808775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:54:46.808775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:54:46.808775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:54:46.808775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 604/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:54:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:46.809405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c55e5-78ab-4f16-b1d5-03a85f807133 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:46.919063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c55e5-78ab-4f16-b1d5-03a85f807133 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:46.944264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:46.959144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531fd35-1c9d-4292-ae91-031a4f5fb824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:46.959814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531fd35-1c9d-4292-ae91-031a4f5fb824 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:46.960096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531fd35-1c9d-4292-ae91-031a4f5fb824 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:46.960367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531fd35-1c9d-4292-ae91-031a4f5fb824 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:46.961207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531fd35-1c9d-4292-ae91-031a4f5fb824 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:46.961521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531fd35-1c9d-4292-ae91-031a4f5fb824 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:46.963790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531fd35-1c9d-4292-ae91-031a4f5fb824 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:46.982599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 614/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:54:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:46.985574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e816b9-8270-4dc1-96bd-ca0a8f43c08c None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:47.103694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e816b9-8270-4dc1-96bd-ca0a8f43c08c None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:47.133116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:47.140112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36240a4a-8ae3-471d-931a-f89eb8967996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:47.140790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36240a4a-8ae3-471d-931a-f89eb8967996 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:47.141162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36240a4a-8ae3-471d-931a-f89eb8967996 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:47.141347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36240a4a-8ae3-471d-931a-f89eb8967996 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:47.142235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36240a4a-8ae3-471d-931a-f89eb8967996 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=bff19e37fa2243fda914f7f4e662c8f7) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:47.142592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36240a4a-8ae3-471d-931a-f89eb8967996 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:47.144141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36240a4a-8ae3-471d-931a-f89eb8967996 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:47.158651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 605/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:54:46 2022] GET /identity/v3/role_assignments?user.id=bff19e37fa2243fda914f7f4e662c8f7&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:49.322593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c667e0-a893-4477-a9b5-8f6339918373 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:49.323156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c667e0-a893-4477-a9b5-8f6339918373 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:49.323156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c667e0-a893-4477-a9b5-8f6339918373 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:49.324602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 615/1220] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:54:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:54:49.330802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e4c5f-507a-494f-add8-353fb3302049 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:49.331334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e4c5f-507a-494f-add8-353fb3302049 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:49.331334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e4c5f-507a-494f-add8-353fb3302049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:49.351310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-782e4c5f-507a-494f-add8-353fb3302049 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:54:49.353497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-782e4c5f-507a-494f-add8-353fb3302049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:54:49.430691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 606/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:54:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:54:49.456444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c667e0-a893-4477-a9b5-8f6339918373 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:49.565969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c667e0-a893-4477-a9b5-8f6339918373 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:49.605016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75aea4-776f-4f19-9454-b44e6854ff61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:49.605844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75aea4-776f-4f19-9454-b44e6854ff61 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:49.606190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75aea4-776f-4f19-9454-b44e6854ff61 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:49.606422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75aea4-776f-4f19-9454-b44e6854ff61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:49.617574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75aea4-776f-4f19-9454-b44e6854ff61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-0d75aea4-776f-4f19-9454-b44e6854ff61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:54:49.626223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:49.627211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:54:49.628044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 616/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:54:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:49.634144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e4c5f-507a-494f-add8-353fb3302049 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:49.758338 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e4c5f-507a-494f-add8-353fb3302049 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:49.784862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:49.797717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e9a25-ca63-48bc-9bb7-ede32f2ffb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:49.798577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e9a25-ca63-48bc-9bb7-ede32f2ffb8b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:49.798855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e9a25-ca63-48bc-9bb7-ede32f2ffb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:49.799327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e9a25-ca63-48bc-9bb7-ede32f2ffb8b None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:49.800320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e9a25-ca63-48bc-9bb7-ede32f2ffb8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:49.800678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e9a25-ca63-48bc-9bb7-ede32f2ffb8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:49.802582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e9a25-ca63-48bc-9bb7-ede32f2ffb8b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:49.811396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 607/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:54:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:49.817301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75aea4-776f-4f19-9454-b44e6854ff61 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:49.865830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:49.933668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75aea4-776f-4f19-9454-b44e6854ff61 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:49.971322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c863d41-349e-4cf8-af97-f48f20676a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:49.972508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c863d41-349e-4cf8-af97-f48f20676a77 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:49.972807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c863d41-349e-4cf8-af97-f48f20676a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:49.973076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c863d41-349e-4cf8-af97-f48f20676a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:49.988939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c863d41-349e-4cf8-af97-f48f20676a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:49.991459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c863d41-349e-4cf8-af97-f48f20676a77 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-9c863d41-349e-4cf8-af97-f48f20676a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:54:50.012149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:50.013240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:50.014430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:54:50.014430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:54:50.014430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:50.014430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:54:50.014430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 617/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:50.019429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e9a25-ca63-48bc-9bb7-ede32f2ffb8b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:50.131421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e9a25-ca63-48bc-9bb7-ede32f2ffb8b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:50.165276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74ddd3-df1e-447e-aba7-808e5458585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:50.165950 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74ddd3-df1e-447e-aba7-808e5458585f None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:50.166267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74ddd3-df1e-447e-aba7-808e5458585f None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:50.166592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74ddd3-df1e-447e-aba7-808e5458585f None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:50.167297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74ddd3-df1e-447e-aba7-808e5458585f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:50.167606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74ddd3-df1e-447e-aba7-808e5458585f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:50.169046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74ddd3-df1e-447e-aba7-808e5458585f None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:50.181608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 608/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:50.187203 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c863d41-349e-4cf8-af97-f48f20676a77 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:50.287626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:50.294961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c863d41-349e-4cf8-af97-f48f20676a77 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:50.330850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bd99c-8eee-4797-9651-02c7ddce2a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:50.331712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bd99c-8eee-4797-9651-02c7ddce2a45 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:50.331991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bd99c-8eee-4797-9651-02c7ddce2a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:50.332263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bd99c-8eee-4797-9651-02c7ddce2a45 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:50.339872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bd99c-8eee-4797-9651-02c7ddce2a45 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:50.341310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bd99c-8eee-4797-9651-02c7ddce2a45 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-c58bd99c-8eee-4797-9651-02c7ddce2a45 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:54:50.356125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:50.357240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:54:50.358208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 618/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:50.363584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74ddd3-df1e-447e-aba7-808e5458585f None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:50.414185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:50.478648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74ddd3-df1e-447e-aba7-808e5458585f None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:50.517767 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7228e4-80b6-4200-b600-1ed5950a1eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:50.518577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7228e4-80b6-4200-b600-1ed5950a1eed None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:50.518935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7228e4-80b6-4200-b600-1ed5950a1eed None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:50.519250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7228e4-80b6-4200-b600-1ed5950a1eed None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:50.520276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7228e4-80b6-4200-b600-1ed5950a1eed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:50.520593 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7228e4-80b6-4200-b600-1ed5950a1eed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:50.522200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7228e4-80b6-4200-b600-1ed5950a1eed None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:50.544554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 609/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jun 20 10:54:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:50.550365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bd99c-8eee-4797-9651-02c7ddce2a45 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:50.670431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bd99c-8eee-4797-9651-02c7ddce2a45 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:50.707969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fa7b1-2ac1-460d-8bfe-64bf369c71bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:50.708784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fa7b1-2ac1-460d-8bfe-64bf369c71bb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:50.709094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fa7b1-2ac1-460d-8bfe-64bf369c71bb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:50.709427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fa7b1-2ac1-460d-8bfe-64bf369c71bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:50.721138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fa7b1-2ac1-460d-8bfe-64bf369c71bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:50.722772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fa7b1-2ac1-460d-8bfe-64bf369c71bb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-416fa7b1-2ac1-460d-8bfe-64bf369c71bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:54:50.742193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:50.743507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:50.744783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:54:50.744783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:54:50.744783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:50.744783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:54:50.744783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 619/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:50.750479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7228e4-80b6-4200-b600-1ed5950a1eed None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:50.861399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7228e4-80b6-4200-b600-1ed5950a1eed None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:50.863919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:50.901960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99fb3e-6dd4-4c93-9634-c2e0e4c782eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:50.902761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99fb3e-6dd4-4c93-9634-c2e0e4c782eb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:50.902951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99fb3e-6dd4-4c93-9634-c2e0e4c782eb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:50.903223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99fb3e-6dd4-4c93-9634-c2e0e4c782eb None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:50.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99fb3e-6dd4-4c93-9634-c2e0e4c782eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:50.904279 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99fb3e-6dd4-4c93-9634-c2e0e4c782eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:50.905680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99fb3e-6dd4-4c93-9634-c2e0e4c782eb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:50.918238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 610/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:54:50.924496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fa7b1-2ac1-460d-8bfe-64bf369c71bb None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:50.935680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:51.037748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fa7b1-2ac1-460d-8bfe-64bf369c71bb None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:51.074976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c8371-ca45-4c5d-a8ad-5646905d9e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:51.075836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c8371-ca45-4c5d-a8ad-5646905d9e26 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:51.076117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c8371-ca45-4c5d-a8ad-5646905d9e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:51.076387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c8371-ca45-4c5d-a8ad-5646905d9e26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:51.083730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c8371-ca45-4c5d-a8ad-5646905d9e26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:51.085175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c8371-ca45-4c5d-a8ad-5646905d9e26 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e56c8371-ca45-4c5d-a8ad-5646905d9e26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:54:51.100065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:51.101231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:51.102647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:54:51.102647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:54:51.102647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:54:51.102647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:54:51.102647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:54:51.102647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:54:51.102647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:54:51.102647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:54:51.102647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 620/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:54:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:51.108607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99fb3e-6dd4-4c93-9634-c2e0e4c782eb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:51.224105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99fb3e-6dd4-4c93-9634-c2e0e4c782eb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:51.263677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95763423-6ba2-4262-b811-e89d6d913da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:51.264426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95763423-6ba2-4262-b811-e89d6d913da1 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:51.264749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95763423-6ba2-4262-b811-e89d6d913da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:51.265026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95763423-6ba2-4262-b811-e89d6d913da1 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:51.265796 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95763423-6ba2-4262-b811-e89d6d913da1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:51.266127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95763423-6ba2-4262-b811-e89d6d913da1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:51.267599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95763423-6ba2-4262-b811-e89d6d913da1 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:51.287410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 611/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:54:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:51.294621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c8371-ca45-4c5d-a8ad-5646905d9e26 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:51.371318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:51.408956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c8371-ca45-4c5d-a8ad-5646905d9e26 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:51.447592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7451c-709d-4202-bfdf-4b1063e2ff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:51.448608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7451c-709d-4202-bfdf-4b1063e2ff53 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:51.448931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7451c-709d-4202-bfdf-4b1063e2ff53 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:51.449301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7451c-709d-4202-bfdf-4b1063e2ff53 None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/bff19e37fa2243fda914f7f4e662c8f7/roles/2cf42fcd695a4696b148241bfd911357` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:51.468845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7451c-709d-4202-bfdf-4b1063e2ff53 None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=bff19e37fa2243fda914f7f4e662c8f7, role_id=2cf42fcd695a4696b148241bfd911357)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:51.470803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7451c-709d-4202-bfdf-4b1063e2ff53 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:51.484071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 621/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:54:51 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/bff19e37fa2243fda914f7f4e662c8f7/roles/2cf42fcd695a4696b148241bfd911357 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:54:53.836545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7e295-1ac0-4403-a440-124c86705df8 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:53.837210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7e295-1ac0-4403-a440-124c86705df8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:53.837210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7e295-1ac0-4403-a440-124c86705df8 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:53.838581 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 612/1233] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:54:53.844553 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6607abf-daea-43cc-a282-98f4944a4d97 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:53.845125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6607abf-daea-43cc-a282-98f4944a4d97 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:53.845218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6607abf-daea-43cc-a282-98f4944a4d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:53.862385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a6607abf-daea-43cc-a282-98f4944a4d97 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:54:53.863676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a6607abf-daea-43cc-a282-98f4944a4d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:54:53.941123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 622/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:54:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:54:53.965380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7e295-1ac0-4403-a440-124c86705df8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:53.994285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:54.082330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7e295-1ac0-4403-a440-124c86705df8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:54.117931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf85ff-da87-4b19-9769-a2d33f62b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:54.118824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf85ff-da87-4b19-9769-a2d33f62b174 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:54.119118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf85ff-da87-4b19-9769-a2d33f62b174 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:54.119399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf85ff-da87-4b19-9769-a2d33f62b174 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:54.130478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf85ff-da87-4b19-9769-a2d33f62b174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-bdbf85ff-da87-4b19-9769-a2d33f62b174 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:54:54.140893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:54.141877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:54:54.143006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 613/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Jun 20 10:54:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:54.148614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6607abf-daea-43cc-a282-98f4944a4d97 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:54.201589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:54.258295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6607abf-daea-43cc-a282-98f4944a4d97 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:54.295635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e74e6-fc9f-4dd5-9579-b98aea7b8791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:54.296556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e74e6-fc9f-4dd5-9579-b98aea7b8791 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:54.296811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e74e6-fc9f-4dd5-9579-b98aea7b8791 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:54.297151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e74e6-fc9f-4dd5-9579-b98aea7b8791 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:54.298224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e74e6-fc9f-4dd5-9579-b98aea7b8791 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:54.298568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e74e6-fc9f-4dd5-9579-b98aea7b8791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:54.300152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e74e6-fc9f-4dd5-9579-b98aea7b8791 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:54.313157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 623/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Jun 20 10:54:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:54.318875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf85ff-da87-4b19-9769-a2d33f62b174 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:54.425883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:54.451133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf85ff-da87-4b19-9769-a2d33f62b174 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:54.488825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e3dce-80a8-4ddd-bae3-357a27119304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:54.489643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e3dce-80a8-4ddd-bae3-357a27119304 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:54.489927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e3dce-80a8-4ddd-bae3-357a27119304 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:54.490242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e3dce-80a8-4ddd-bae3-357a27119304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:54.501818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e3dce-80a8-4ddd-bae3-357a27119304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:54.503321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e3dce-80a8-4ddd-bae3-357a27119304 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-4a8e3dce-80a8-4ddd-bae3-357a27119304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:54:54.521036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:54.522128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:54.523191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:54:54.523191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:54:54.523191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:54.523191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:54:54.523191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 614/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:54:54.528260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e74e6-fc9f-4dd5-9579-b98aea7b8791 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:54.641474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e74e6-fc9f-4dd5-9579-b98aea7b8791 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:54.655033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:54.684400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacd9ec-f4d0-47df-8f0b-3320e3faea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:54.684993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacd9ec-f4d0-47df-8f0b-3320e3faea38 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:54.685299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacd9ec-f4d0-47df-8f0b-3320e3faea38 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:54.685540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacd9ec-f4d0-47df-8f0b-3320e3faea38 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:54.686288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cacd9ec-f4d0-47df-8f0b-3320e3faea38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:54.686600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cacd9ec-f4d0-47df-8f0b-3320e3faea38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:54.687965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cacd9ec-f4d0-47df-8f0b-3320e3faea38 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:54.700323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 624/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:54.706614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e3dce-80a8-4ddd-bae3-357a27119304 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:54.817644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e3dce-80a8-4ddd-bae3-357a27119304 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:54.857123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19c9f8-f34c-4d9d-84ba-5b46340a3a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:54.858063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19c9f8-f34c-4d9d-84ba-5b46340a3a60 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:54.858428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19c9f8-f34c-4d9d-84ba-5b46340a3a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:54.858810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19c9f8-f34c-4d9d-84ba-5b46340a3a60 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:54.862438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:54.869563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19c9f8-f34c-4d9d-84ba-5b46340a3a60 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:54.871919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19c9f8-f34c-4d9d-84ba-5b46340a3a60 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-7b19c9f8-f34c-4d9d-84ba-5b46340a3a60 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:54:54.889231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:54.890551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:54:54.892111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 615/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:54.896751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacd9ec-f4d0-47df-8f0b-3320e3faea38 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:55.003050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacd9ec-f4d0-47df-8f0b-3320e3faea38 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:55.037452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9b9e9-4858-45a4-baec-bb065d400ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:55.038155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9b9e9-4858-45a4-baec-bb065d400ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:55.038434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9b9e9-4858-45a4-baec-bb065d400ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:55.038707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9b9e9-4858-45a4-baec-bb065d400ba3 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:55.039542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9b9e9-4858-45a4-baec-bb065d400ba3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:55.039898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9b9e9-4858-45a4-baec-bb065d400ba3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:55.041325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9b9e9-4858-45a4-baec-bb065d400ba3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:55.059366 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 625/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jun 20 10:54:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:55.065541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19c9f8-f34c-4d9d-84ba-5b46340a3a60 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:55.172485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19c9f8-f34c-4d9d-84ba-5b46340a3a60 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:55.209044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c414b81-d5c0-428f-88b5-7917256938fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:55.209751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414b81-d5c0-428f-88b5-7917256938fb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:55.210076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414b81-d5c0-428f-88b5-7917256938fb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:55.210383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414b81-d5c0-428f-88b5-7917256938fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:55.274385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:55.281814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c414b81-d5c0-428f-88b5-7917256938fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:55.283449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c414b81-d5c0-428f-88b5-7917256938fb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:55.390789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 616/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:54:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 10:54:55.396957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9b9e9-4858-45a4-baec-bb065d400ba3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:55.428879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:55.517255 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9b9e9-4858-45a4-baec-bb065d400ba3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:55.556841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c675fc5-a753-4aae-b8bb-e4c6063f707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:55.557654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675fc5-a753-4aae-b8bb-e4c6063f707f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:55.557935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675fc5-a753-4aae-b8bb-e4c6063f707f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:55.558243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675fc5-a753-4aae-b8bb-e4c6063f707f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:55.568841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c675fc5-a753-4aae-b8bb-e4c6063f707f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:55.570390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c675fc5-a753-4aae-b8bb-e4c6063f707f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2c675fc5-a753-4aae-b8bb-e4c6063f707f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:54:55.589335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:55.590522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:55.591453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:54:55.591453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:54:55.591453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:55.591453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:54:55.591453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 626/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:55.596667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c414b81-d5c0-428f-88b5-7917256938fb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:55.711042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:55.717590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c414b81-d5c0-428f-88b5-7917256938fb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:55.755464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621bc50-9eb7-4efd-ac53-d857a173eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:55.756227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621bc50-9eb7-4efd-ac53-d857a173eb91 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:55.756510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621bc50-9eb7-4efd-ac53-d857a173eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:55.756823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621bc50-9eb7-4efd-ac53-d857a173eb91 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:55.757623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621bc50-9eb7-4efd-ac53-d857a173eb91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:55.757930 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621bc50-9eb7-4efd-ac53-d857a173eb91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:55.760210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621bc50-9eb7-4efd-ac53-d857a173eb91 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:55.772373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 617/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:55.779051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c675fc5-a753-4aae-b8bb-e4c6063f707f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:55.869222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:55.893492 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c675fc5-a753-4aae-b8bb-e4c6063f707f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:55.937663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993b04d-4c68-49f9-bd0b-19c0c5204a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:55.938499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993b04d-4c68-49f9-bd0b-19c0c5204a7d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:55.938778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993b04d-4c68-49f9-bd0b-19c0c5204a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:55.939092 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993b04d-4c68-49f9-bd0b-19c0c5204a7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:55.945954 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993b04d-4c68-49f9-bd0b-19c0c5204a7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:55.947517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993b04d-4c68-49f9-bd0b-19c0c5204a7d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e993b04d-4c68-49f9-bd0b-19c0c5204a7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:54:55.962913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:55.963971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:55.965141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:54:55.965141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:54:55.965141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:54:55.965141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:54:55.965141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:54:55.965141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:54:55.965141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:54:55.965141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:54:55.965141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 627/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:54:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:55.970792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621bc50-9eb7-4efd-ac53-d857a173eb91 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:56.106021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621bc50-9eb7-4efd-ac53-d857a173eb91 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:56.145515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60b9c4-2a6e-4556-ae3b-d3320e6fdc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:56.146261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60b9c4-2a6e-4556-ae3b-d3320e6fdc57 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:56.146588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60b9c4-2a6e-4556-ae3b-d3320e6fdc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:56.146937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60b9c4-2a6e-4556-ae3b-d3320e6fdc57 None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:56.148016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60b9c4-2a6e-4556-ae3b-d3320e6fdc57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:56.148421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60b9c4-2a6e-4556-ae3b-d3320e6fdc57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:56.150259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60b9c4-2a6e-4556-ae3b-d3320e6fdc57 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:56.162936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 618/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:54:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:56.170746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993b04d-4c68-49f9-bd0b-19c0c5204a7d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:56.287593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993b04d-4c68-49f9-bd0b-19c0c5204a7d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:56.326567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096c611-0763-47c4-81cc-f36963d9c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:56.327237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096c611-0763-47c4-81cc-f36963d9c365 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:56.327519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096c611-0763-47c4-81cc-f36963d9c365 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:56.327795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096c611-0763-47c4-81cc-f36963d9c365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:56.328994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096c611-0763-47c4-81cc-f36963d9c365 None admin] RBAC: Adding query filter params (role.id=2cf42fcd695a4696b148241bfd911357, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=bff19e37fa2243fda914f7f4e662c8f7) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:56.329309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096c611-0763-47c4-81cc-f36963d9c365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:56.330798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096c611-0763-47c4-81cc-f36963d9c365 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:56.338383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 628/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:54:56 2022] GET /identity/v3/role_assignments?user.id=bff19e37fa2243fda914f7f4e662c8f7&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=2cf42fcd695a4696b148241bfd911357 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:58.418690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da0910-717c-416f-9154-aabff2d79547 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:58.419255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da0910-717c-416f-9154-aabff2d79547 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:58.419380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da0910-717c-416f-9154-aabff2d79547 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:58.420894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 619/1247] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:54:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:54:58.426409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc645e26-0b9b-47ea-b5ba-0082b302a3c3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:58.426851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc645e26-0b9b-47ea-b5ba-0082b302a3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:58.427036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc645e26-0b9b-47ea-b5ba-0082b302a3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:58.432571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:58.448035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bc645e26-0b9b-47ea-b5ba-0082b302a3c3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:54:58.449749 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bc645e26-0b9b-47ea-b5ba-0082b302a3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:54:58.526124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 629/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:54:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:54:58.553719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da0910-717c-416f-9154-aabff2d79547 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:58.577273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:58.679327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da0910-717c-416f-9154-aabff2d79547 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:58.723687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd4bbc-dcfc-47c4-b13b-c752b28e040b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:58.724526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4bbc-dcfc-47c4-b13b-c752b28e040b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:58.724808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4bbc-dcfc-47c4-b13b-c752b28e040b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:58.725076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd4bbc-dcfc-47c4-b13b-c752b28e040b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:58.736485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd4bbc-dcfc-47c4-b13b-c752b28e040b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-6fdd4bbc-dcfc-47c4-b13b-c752b28e040b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:54:58.744714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:58.745795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:54:58.746681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 620/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:54:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:58.753088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc645e26-0b9b-47ea-b5ba-0082b302a3c3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:58.862898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc645e26-0b9b-47ea-b5ba-0082b302a3c3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:58.894842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:58.901820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e46303-6b1e-4cdc-aa19-85876b43d94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:58.902540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e46303-6b1e-4cdc-aa19-85876b43d94c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:58.902837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e46303-6b1e-4cdc-aa19-85876b43d94c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:58.903077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e46303-6b1e-4cdc-aa19-85876b43d94c None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:58.903826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e46303-6b1e-4cdc-aa19-85876b43d94c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:58.904131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e46303-6b1e-4cdc-aa19-85876b43d94c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:58.905510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e46303-6b1e-4cdc-aa19-85876b43d94c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:58.914983 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 630/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:54:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:58.921128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd4bbc-dcfc-47c4-b13b-c752b28e040b None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:59.013080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:59.038671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd4bbc-dcfc-47c4-b13b-c752b28e040b None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:59.074177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494ec56-afa0-40c4-b735-7940e2e134cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:59.074980 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494ec56-afa0-40c4-b735-7940e2e134cd None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:59.075256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494ec56-afa0-40c4-b735-7940e2e134cd None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:59.075528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494ec56-afa0-40c4-b735-7940e2e134cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:59.088428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494ec56-afa0-40c4-b735-7940e2e134cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:59.089895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494ec56-afa0-40c4-b735-7940e2e134cd None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-3494ec56-afa0-40c4-b735-7940e2e134cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:54:59.108152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:59.109434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:59.110416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:54:59.110416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:54:59.110416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:54:59.110416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:54:59.110416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 621/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:59.115562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e46303-6b1e-4cdc-aa19-85876b43d94c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:59.219198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e46303-6b1e-4cdc-aa19-85876b43d94c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:59.254650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed2bca-bbb2-4fa7-a3b1-a98e4dc889d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:59.255326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed2bca-bbb2-4fa7-a3b1-a98e4dc889d6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:59.255621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed2bca-bbb2-4fa7-a3b1-a98e4dc889d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:59.256156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed2bca-bbb2-4fa7-a3b1-a98e4dc889d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:59.256616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed2bca-bbb2-4fa7-a3b1-a98e4dc889d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:59.256888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed2bca-bbb2-4fa7-a3b1-a98e4dc889d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:59.258282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed2bca-bbb2-4fa7-a3b1-a98e4dc889d6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:59.270485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 631/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:59.276775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494ec56-afa0-40c4-b735-7940e2e134cd None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:59.383375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494ec56-afa0-40c4-b735-7940e2e134cd None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:59.410842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:59.420296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c788d4-5028-4a97-9c0c-66614b604bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:59.421138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c788d4-5028-4a97-9c0c-66614b604bab None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:59.421424 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c788d4-5028-4a97-9c0c-66614b604bab None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:59.421699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c788d4-5028-4a97-9c0c-66614b604bab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:59.430188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c788d4-5028-4a97-9c0c-66614b604bab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:59.431889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c788d4-5028-4a97-9c0c-66614b604bab None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-46c788d4-5028-4a97-9c0c-66614b604bab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:54:59.446694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:54:59.447910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:54:59.449122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 622/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:54:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:54:59.454572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed2bca-bbb2-4fa7-a3b1-a98e4dc889d6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:59.516687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:59.564868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed2bca-bbb2-4fa7-a3b1-a98e4dc889d6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:59.603427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795d5f7-75cf-469f-886e-00f0f2488f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:59.604073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795d5f7-75cf-469f-886e-00f0f2488f24 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:59.604357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795d5f7-75cf-469f-886e-00f0f2488f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:59.604645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a795d5f7-75cf-469f-886e-00f0f2488f24 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:59.605472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795d5f7-75cf-469f-886e-00f0f2488f24 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:54:59.605855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795d5f7-75cf-469f-886e-00f0f2488f24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:59.607319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a795d5f7-75cf-469f-886e-00f0f2488f24 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:54:59.632387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 632/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jun 20 10:54:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:54:59.633836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c788d4-5028-4a97-9c0c-66614b604bab None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:54:59.759600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c788d4-5028-4a97-9c0c-66614b604bab None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:54:59.800600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b5148-f591-40ca-9523-9b82a1f6b9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:54:59.801251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b5148-f591-40ca-9523-9b82a1f6b9be None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:54:59.801530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b5148-f591-40ca-9523-9b82a1f6b9be None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:54:59.801802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b5148-f591-40ca-9523-9b82a1f6b9be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:54:59.872785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:54:59.887918 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b5148-f591-40ca-9523-9b82a1f6b9be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:54:59.889568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b5148-f591-40ca-9523-9b82a1f6b9be None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:00.008507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 623/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:54:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 10:55:00.014817 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795d5f7-75cf-469f-886e-00f0f2488f24 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:00.121609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a795d5f7-75cf-469f-886e-00f0f2488f24 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:00.137005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:00.162067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bf6f2-ecf2-4e8b-942c-6e1f14c16b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:00.162910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bf6f2-ecf2-4e8b-942c-6e1f14c16b70 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:00.163176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bf6f2-ecf2-4e8b-942c-6e1f14c16b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:00.163451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bf6f2-ecf2-4e8b-942c-6e1f14c16b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:00.174660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bf6f2-ecf2-4e8b-942c-6e1f14c16b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:00.176236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bf6f2-ecf2-4e8b-942c-6e1f14c16b70 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-3f2bf6f2-ecf2-4e8b-942c-6e1f14c16b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:55:00.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:00.195580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:00.196831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:55:00.196831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:55:00.196831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:55:00.196831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:00.196831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 633/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:00.203858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b5148-f591-40ca-9523-9b82a1f6b9be None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:00.303793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:00.316547 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b5148-f591-40ca-9523-9b82a1f6b9be None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:00.359314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045ec39-aa90-450c-961a-f305bbeae621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:00.360037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045ec39-aa90-450c-961a-f305bbeae621 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:00.360296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045ec39-aa90-450c-961a-f305bbeae621 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:00.360562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045ec39-aa90-450c-961a-f305bbeae621 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:00.361320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045ec39-aa90-450c-961a-f305bbeae621 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:00.361599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045ec39-aa90-450c-961a-f305bbeae621 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:00.363015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045ec39-aa90-450c-961a-f305bbeae621 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:00.375322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 624/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:00.381211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bf6f2-ecf2-4e8b-942c-6e1f14c16b70 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:00.491390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bf6f2-ecf2-4e8b-942c-6e1f14c16b70 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:00.530448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cbe5b-dbb9-4817-aa1b-cec7ffaeee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:00.531394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cbe5b-dbb9-4817-aa1b-cec7ffaeee08 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:00.531672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cbe5b-dbb9-4817-aa1b-cec7ffaeee08 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:00.532088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cbe5b-dbb9-4817-aa1b-cec7ffaeee08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:00.541092 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cbe5b-dbb9-4817-aa1b-cec7ffaeee08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:00.542970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cbe5b-dbb9-4817-aa1b-cec7ffaeee08 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-f79cbe5b-dbb9-4817-aa1b-cec7ffaeee08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:55:00.558339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:00.559450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:00.560642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:55:00.560642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:55:00.560642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:55:00.560642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:55:00.560642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:55:00.560642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:55:00.560642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:55:00.560642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:00.560642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 634/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:55:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:55:00.566044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045ec39-aa90-450c-961a-f305bbeae621 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:00.674506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045ec39-aa90-450c-961a-f305bbeae621 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:00.716151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d5c9e-67d2-4199-a575-255ef370214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:00.716958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d5c9e-67d2-4199-a575-255ef370214b None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:00.717245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d5c9e-67d2-4199-a575-255ef370214b None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:00.717526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d5c9e-67d2-4199-a575-255ef370214b None admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:00.718388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d5c9e-67d2-4199-a575-255ef370214b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:00.718671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d5c9e-67d2-4199-a575-255ef370214b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:00.720469 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d5c9e-67d2-4199-a575-255ef370214b None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:00.733808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 625/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:55:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:00.740174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cbe5b-dbb9-4817-aa1b-cec7ffaeee08 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:00.768768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:00.858069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cbe5b-dbb9-4817-aa1b-cec7ffaeee08 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:00.897370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54163fd-a201-45a6-b337-d241c2f9d951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:00.898030 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54163fd-a201-45a6-b337-d241c2f9d951 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:00.898370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54163fd-a201-45a6-b337-d241c2f9d951 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:00.898635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54163fd-a201-45a6-b337-d241c2f9d951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:00.900078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54163fd-a201-45a6-b337-d241c2f9d951 None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=bff19e37fa2243fda914f7f4e662c8f7) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:00.900392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54163fd-a201-45a6-b337-d241c2f9d951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:00.901858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54163fd-a201-45a6-b337-d241c2f9d951 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:00.909830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 635/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:55:00 2022] GET /identity/v3/role_assignments?user.id=bff19e37fa2243fda914f7f4e662c8f7&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:55:02.989191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e2ea-00a2-4c01-a0e9-22e8381335c4 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:02.990016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e2ea-00a2-4c01-a0e9-22e8381335c4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:02.990016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478e2ea-00a2-4c01-a0e9-22e8381335c4 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:02.992319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 626/1261] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:55:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:55:02.997828 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa256a-1281-4065-8f0a-08cd9d398a1a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:02.998263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa256a-1281-4065-8f0a-08cd9d398a1a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:02.998409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa256a-1281-4065-8f0a-08cd9d398a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:03.016789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d8fa256a-1281-4065-8f0a-08cd9d398a1a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:55:03.017967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d8fa256a-1281-4065-8f0a-08cd9d398a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:55:03.086382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 636/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:55:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:55:03.109452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e2ea-00a2-4c01-a0e9-22e8381335c4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:03.121937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:03.222066 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478e2ea-00a2-4c01-a0e9-22e8381335c4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:03.263157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8379-5b67-4e80-b194-9b02f2a9dad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:03.263955 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8379-5b67-4e80-b194-9b02f2a9dad0 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:03.264240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8379-5b67-4e80-b194-9b02f2a9dad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:03.264525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8379-5b67-4e80-b194-9b02f2a9dad0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:03.275760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b8379-5b67-4e80-b194-9b02f2a9dad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-5c1b8379-5b67-4e80-b194-9b02f2a9dad0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:55:03.283297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:03.284451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:55:03.290554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 627/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:55:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:03.291511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa256a-1281-4065-8f0a-08cd9d398a1a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:03.315089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:03.396341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa256a-1281-4065-8f0a-08cd9d398a1a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:03.435585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29d2bd-1e7f-4ba1-977b-d745cd93b26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:03.436246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29d2bd-1e7f-4ba1-977b-d745cd93b26d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:03.436522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29d2bd-1e7f-4ba1-977b-d745cd93b26d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:03.436794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29d2bd-1e7f-4ba1-977b-d745cd93b26d None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:03.437544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29d2bd-1e7f-4ba1-977b-d745cd93b26d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:03.437848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29d2bd-1e7f-4ba1-977b-d745cd93b26d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:03.439269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29d2bd-1e7f-4ba1-977b-d745cd93b26d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:03.447481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 637/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:55:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:03.454487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8379-5b67-4e80-b194-9b02f2a9dad0 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:03.543016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:03.578905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8379-5b67-4e80-b194-9b02f2a9dad0 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:03.616715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab5dc1-f0eb-407b-80c1-a434a0fd2104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:03.617522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab5dc1-f0eb-407b-80c1-a434a0fd2104 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:03.618010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab5dc1-f0eb-407b-80c1-a434a0fd2104 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:03.618373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab5dc1-f0eb-407b-80c1-a434a0fd2104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:03.630833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab5dc1-f0eb-407b-80c1-a434a0fd2104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:03.632294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab5dc1-f0eb-407b-80c1-a434a0fd2104 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-42ab5dc1-f0eb-407b-80c1-a434a0fd2104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:55:03.654188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:03.655615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:03.656690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:55:03.656690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:55:03.656690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:55:03.656690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:55:03.656690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 628/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:03.662124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29d2bd-1e7f-4ba1-977b-d745cd93b26d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:03.764796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:03.786109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29d2bd-1e7f-4ba1-977b-d745cd93b26d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:03.825205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce8335-4716-44c9-a529-12cec61299bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:03.825798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce8335-4716-44c9-a529-12cec61299bc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:03.826077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce8335-4716-44c9-a529-12cec61299bc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:03.826386 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce8335-4716-44c9-a529-12cec61299bc None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:03.827098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce8335-4716-44c9-a529-12cec61299bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:03.827406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce8335-4716-44c9-a529-12cec61299bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:03.829145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce8335-4716-44c9-a529-12cec61299bc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:03.841096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 638/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:03.847590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab5dc1-f0eb-407b-80c1-a434a0fd2104 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:03.962021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab5dc1-f0eb-407b-80c1-a434a0fd2104 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:03.983344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:03.998647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02542c28-2c2f-4828-b682-6205455c0336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:03.999459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02542c28-2c2f-4828-b682-6205455c0336 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:03.999738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02542c28-2c2f-4828-b682-6205455c0336 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:04.000010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02542c28-2c2f-4828-b682-6205455c0336 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:04.007543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02542c28-2c2f-4828-b682-6205455c0336 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:04.009216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02542c28-2c2f-4828-b682-6205455c0336 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-02542c28-2c2f-4828-b682-6205455c0336 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:55:04.023742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:04.025079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:55:04.028703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 629/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:55:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:04.034449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce8335-4716-44c9-a529-12cec61299bc None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:04.163844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce8335-4716-44c9-a529-12cec61299bc None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:04.200807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742b08a-cd6f-4528-b28c-d53c1ab682b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:04.201471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742b08a-cd6f-4528-b28c-d53c1ab682b2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:04.201751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742b08a-cd6f-4528-b28c-d53c1ab682b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:04.202023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742b08a-cd6f-4528-b28c-d53c1ab682b2 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:04.202896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742b08a-cd6f-4528-b28c-d53c1ab682b2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:04.203576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742b08a-cd6f-4528-b28c-d53c1ab682b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:04.205007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742b08a-cd6f-4528-b28c-d53c1ab682b2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:04.213803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:04.228823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 639/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jun 20 10:55:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:04.238397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02542c28-2c2f-4828-b682-6205455c0336 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:04.345989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02542c28-2c2f-4828-b682-6205455c0336 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:04.381896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7038e-3f51-4bfd-83d0-faa5729ba2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:04.382707 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7038e-3f51-4bfd-83d0-faa5729ba2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:04.382987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7038e-3f51-4bfd-83d0-faa5729ba2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:04.383265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7038e-3f51-4bfd-83d0-faa5729ba2c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:04.394791 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7038e-3f51-4bfd-83d0-faa5729ba2c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:04.396247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7038e-3f51-4bfd-83d0-faa5729ba2c7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-48d7038e-3f51-4bfd-83d0-faa5729ba2c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:55:04.420755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:04.421802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:04.422837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:55:04.422837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:55:04.422837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:55:04.422837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:55:04.422837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 630/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:04.428559 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742b08a-cd6f-4528-b28c-d53c1ab682b2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:04.540452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742b08a-cd6f-4528-b28c-d53c1ab682b2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:04.578842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e555ca-9f28-4ce7-afae-a1a9f1ff1845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:04.579495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e555ca-9f28-4ce7-afae-a1a9f1ff1845 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:04.579773 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e555ca-9f28-4ce7-afae-a1a9f1ff1845 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:04.580043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e555ca-9f28-4ce7-afae-a1a9f1ff1845 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:04.580762 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e555ca-9f28-4ce7-afae-a1a9f1ff1845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:04.581069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e555ca-9f28-4ce7-afae-a1a9f1ff1845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:04.582477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e555ca-9f28-4ce7-afae-a1a9f1ff1845 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:04.594862 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 640/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:04.602562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7038e-3f51-4bfd-83d0-faa5729ba2c7 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:04.642151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:04.720311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7038e-3f51-4bfd-83d0-faa5729ba2c7 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:04.757981 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3bbae-2b43-4e9a-8732-f21edf3ae204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:04.759040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3bbae-2b43-4e9a-8732-f21edf3ae204 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:04.759318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3bbae-2b43-4e9a-8732-f21edf3ae204 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:04.759598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3bbae-2b43-4e9a-8732-f21edf3ae204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:04.766408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3bbae-2b43-4e9a-8732-f21edf3ae204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:04.767869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3bbae-2b43-4e9a-8732-f21edf3ae204 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-29e3bbae-2b43-4e9a-8732-f21edf3ae204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:55:04.782864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:04.784004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:04.785169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:55:04.785169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:55:04.785169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:55:04.785169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:55:04.785169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:55:04.785169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:55:04.785169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:55:04.785169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:55:04.785169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 631/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:55:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:04.791455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e555ca-9f28-4ce7-afae-a1a9f1ff1845 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:04.861171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:04.921922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e555ca-9f28-4ce7-afae-a1a9f1ff1845 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:04.971366 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0f5a8-12d4-47d7-b179-d192119c7aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:04.971961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0f5a8-12d4-47d7-b179-d192119c7aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:04.972245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0f5a8-12d4-47d7-b179-d192119c7aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:04.972523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0f5a8-12d4-47d7-b179-d192119c7aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:04.973323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0f5a8-12d4-47d7-b179-d192119c7aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:04.973636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0f5a8-12d4-47d7-b179-d192119c7aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:04.975459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0f5a8-12d4-47d7-b179-d192119c7aa7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:04.993400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 641/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 10:55:04.999834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3bbae-2b43-4e9a-8732-f21edf3ae204 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:05.118939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:05.128495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3bbae-2b43-4e9a-8732-f21edf3ae204 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:05.178742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70537da8-92fd-4310-a27e-abe04a624386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:05.179662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70537da8-92fd-4310-a27e-abe04a624386 None admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:05.179945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70537da8-92fd-4310-a27e-abe04a624386 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:05.180232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70537da8-92fd-4310-a27e-abe04a624386 None admin] PATH_INFO: `/v3/projects/c904620b9191403ca36d0aca020db817/users/bff19e37fa2243fda914f7f4e662c8f7/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:05.182874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70537da8-92fd-4310-a27e-abe04a624386 None admin] RBAC: Authorizing `identity:create_grant(project_id=c904620b9191403ca36d0aca020db817, user_id=bff19e37fa2243fda914f7f4e662c8f7, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:05.184332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70537da8-92fd-4310-a27e-abe04a624386 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:05.206562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 632/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Jun 20 10:55:04 2022] PUT /identity/v3/projects/c904620b9191403ca36d0aca020db817/users/bff19e37fa2243fda914f7f4e662c8f7/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 10:55:07.333359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79a0ae-4a61-4408-b4c2-b0be69578f4e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:07.333961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79a0ae-4a61-4408-b4c2-b0be69578f4e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:07.333961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79a0ae-4a61-4408-b4c2-b0be69578f4e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:07.335325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 642/1274] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:55:07.340981 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d9d11-6daf-448d-9c40-9039e581cce1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:07.341402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d9d11-6daf-448d-9c40-9039e581cce1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:07.341532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d9d11-6daf-448d-9c40-9039e581cce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:07.368736 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d35d9d11-6daf-448d-9c40-9039e581cce1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:55:07.369919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d35d9d11-6daf-448d-9c40-9039e581cce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:55:07.452226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 633/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:55:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:55:07.476742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79a0ae-4a61-4408-b4c2-b0be69578f4e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:07.585315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79a0ae-4a61-4408-b4c2-b0be69578f4e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:07.602753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:07.621224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94820c75-684c-425d-bf55-c825f712c6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:07.622017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94820c75-684c-425d-bf55-c825f712c6da None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:07.622340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94820c75-684c-425d-bf55-c825f712c6da None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:07.622612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94820c75-684c-425d-bf55-c825f712c6da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:07.634125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94820c75-684c-425d-bf55-c825f712c6da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-94820c75-684c-425d-bf55-c825f712c6da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 10:55:07.641839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:07.643619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:07.644672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 643/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Jun 20 10:55:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:07.649234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d9d11-6daf-448d-9c40-9039e581cce1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:07.733265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:07.755622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d9d11-6daf-448d-9c40-9039e581cce1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:07.791053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc14c83-47f0-4610-bab3-175e202e9c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:07.791717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc14c83-47f0-4610-bab3-175e202e9c00 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:07.791999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc14c83-47f0-4610-bab3-175e202e9c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:07.792271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc14c83-47f0-4610-bab3-175e202e9c00 None admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:07.793028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc14c83-47f0-4610-bab3-175e202e9c00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:07.793334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc14c83-47f0-4610-bab3-175e202e9c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:07.794746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc14c83-47f0-4610-bab3-175e202e9c00 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:07.803307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 634/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Jun 20 10:55:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:07.809367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94820c75-684c-425d-bf55-c825f712c6da None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:07.924279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94820c75-684c-425d-bf55-c825f712c6da None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:07.963577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98898880-c709-4f61-8a99-e77475510b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:07.964367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98898880-c709-4f61-8a99-e77475510b03 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:07.964650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98898880-c709-4f61-8a99-e77475510b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:07.964922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98898880-c709-4f61-8a99-e77475510b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:07.975369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98898880-c709-4f61-8a99-e77475510b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:07.976835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98898880-c709-4f61-8a99-e77475510b03 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-98898880-c709-4f61-8a99-e77475510b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:55:07.994628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:07.995628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:07.996591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:55:07.996591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:55:07.996591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:55:07.996591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:07.996591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 644/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:08.001754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc14c83-47f0-4610-bab3-175e202e9c00 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:08.117374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc14c83-47f0-4610-bab3-175e202e9c00 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:08.152092 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dbd8d-e2c3-4a4b-9a8e-79c407b85555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:08.152745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dbd8d-e2c3-4a4b-9a8e-79c407b85555 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:08.153030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dbd8d-e2c3-4a4b-9a8e-79c407b85555 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:08.153305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dbd8d-e2c3-4a4b-9a8e-79c407b85555 None admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:08.154022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2dbd8d-e2c3-4a4b-9a8e-79c407b85555 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:08.154363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2dbd8d-e2c3-4a4b-9a8e-79c407b85555 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:08.155753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2dbd8d-e2c3-4a4b-9a8e-79c407b85555 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:08.167969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 635/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:08.176251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98898880-c709-4f61-8a99-e77475510b03 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:08.211918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:08.284730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98898880-c709-4f61-8a99-e77475510b03 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:08.325820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa59c9a-d852-42a7-81f3-0fd4c664c472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:08.326663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa59c9a-d852-42a7-81f3-0fd4c664c472 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:08.326945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa59c9a-d852-42a7-81f3-0fd4c664c472 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:08.327215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa59c9a-d852-42a7-81f3-0fd4c664c472 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:08.335211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa59c9a-d852-42a7-81f3-0fd4c664c472 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:08.336772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa59c9a-d852-42a7-81f3-0fd4c664c472 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-caa59c9a-d852-42a7-81f3-0fd4c664c472 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 10:55:08.351229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:08.352405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:08.353682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 645/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:55:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 10:55:08.359356 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dbd8d-e2c3-4a4b-9a8e-79c407b85555 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:08.365559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:08.479486 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dbd8d-e2c3-4a4b-9a8e-79c407b85555 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:08.518066 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56978064-8b87-4736-94c8-08a282cc43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:08.518752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56978064-8b87-4736-94c8-08a282cc43cb None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:08.519032 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56978064-8b87-4736-94c8-08a282cc43cb None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:08.519305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56978064-8b87-4736-94c8-08a282cc43cb None admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:08.520141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56978064-8b87-4736-94c8-08a282cc43cb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:08.520443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56978064-8b87-4736-94c8-08a282cc43cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:08.521891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56978064-8b87-4736-94c8-08a282cc43cb None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:08.539778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 636/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Jun 20 10:55:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:08.548054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa59c9a-d852-42a7-81f3-0fd4c664c472 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:08.644186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:08.657190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa59c9a-d852-42a7-81f3-0fd4c664c472 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:08.705224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81c346-b9f5-4bce-a5a6-622c57aafe3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:08.706156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81c346-b9f5-4bce-a5a6-622c57aafe3f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:08.706519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81c346-b9f5-4bce-a5a6-622c57aafe3f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:08.706896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81c346-b9f5-4bce-a5a6-622c57aafe3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:08.781041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81c346-b9f5-4bce-a5a6-622c57aafe3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:08.782522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81c346-b9f5-4bce-a5a6-622c57aafe3f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:08.874859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:08.885644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 646/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Jun 20 10:55:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 10:55:08.892352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56978064-8b87-4736-94c8-08a282cc43cb None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:08.962243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:09.001374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56978064-8b87-4736-94c8-08a282cc43cb None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:09.036960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a598ee-88f8-4f1a-ae06-2be7bd579829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:09.037677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a598ee-88f8-4f1a-ae06-2be7bd579829 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:09.037962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a598ee-88f8-4f1a-ae06-2be7bd579829 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:09.038272 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a598ee-88f8-4f1a-ae06-2be7bd579829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:09.048993 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a598ee-88f8-4f1a-ae06-2be7bd579829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:09.050496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a598ee-88f8-4f1a-ae06-2be7bd579829 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-e6a598ee-88f8-4f1a-ae06-2be7bd579829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 10:55:09.067895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:09.068907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:09.069969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 10:55:09.069969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 10:55:09.069969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 10:55:09.069969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:55:09.069969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 637/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:09.075292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81c346-b9f5-4bce-a5a6-622c57aafe3f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:09.177985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81c346-b9f5-4bce-a5a6-622c57aafe3f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:09.213851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41737618-a75f-4987-8d0f-589317ec64d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:09.214536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41737618-a75f-4987-8d0f-589317ec64d2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:09.214817 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41737618-a75f-4987-8d0f-589317ec64d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:09.215090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41737618-a75f-4987-8d0f-589317ec64d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:09.215809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41737618-a75f-4987-8d0f-589317ec64d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:09.216114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41737618-a75f-4987-8d0f-589317ec64d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:09.217507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41737618-a75f-4987-8d0f-589317ec64d2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:09.229640 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 647/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Jun 20 10:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:09.235499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a598ee-88f8-4f1a-ae06-2be7bd579829 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:09.350523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a598ee-88f8-4f1a-ae06-2be7bd579829 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:09.365498 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:09.388790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cf4b9-07f0-4e3d-800f-2a1f8e3dd031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:09.390013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cf4b9-07f0-4e3d-800f-2a1f8e3dd031 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:09.390483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cf4b9-07f0-4e3d-800f-2a1f8e3dd031 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:09.390917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cf4b9-07f0-4e3d-800f-2a1f8e3dd031 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:09.401236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54cf4b9-07f0-4e3d-800f-2a1f8e3dd031 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:09.402746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54cf4b9-07f0-4e3d-800f-2a1f8e3dd031 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application [None req-c54cf4b9-07f0-4e3d-800f-2a1f8e3dd031 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 10:55:09.416716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:09.417700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:09.418794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 10:55:09.418794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 10:55:09.418794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 10:55:09.418794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 10:55:09.418794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 10:55:09.418794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 10:55:09.418794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 10:55:09.418794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: ERROR keystone.server.flask.application Jun 20 10:55:09.418794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 638/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Jun 20 10:55:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:09.423902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41737618-a75f-4987-8d0f-589317ec64d2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:09.480044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:09.535458 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41737618-a75f-4987-8d0f-589317ec64d2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:09.569897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee889c9c-ce15-46ad-adf4-214414f355e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:09.570572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889c9c-ce15-46ad-adf4-214414f355e9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:09.570890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889c9c-ce15-46ad-adf4-214414f355e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:09.571152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889c9c-ce15-46ad-adf4-214414f355e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:09.571933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889c9c-ce15-46ad-adf4-214414f355e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:09.572242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889c9c-ce15-46ad-adf4-214414f355e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:09.573672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889c9c-ce15-46ad-adf4-214414f355e9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:09.586011 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 648/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Jun 20 10:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:09.592565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cf4b9-07f0-4e3d-800f-2a1f8e3dd031 None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:09.698835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cf4b9-07f0-4e3d-800f-2a1f8e3dd031 None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:09.733277 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a6862-2634-4cc8-b61f-64c64881f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:09.733921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a6862-2634-4cc8-b61f-64c64881f5aa None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:09.734230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a6862-2634-4cc8-b61f-64c64881f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:09.734504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a6862-2634-4cc8-b61f-64c64881f5aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:09.735329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a6862-2634-4cc8-b61f-64c64881f5aa None admin] RBAC: Adding query filter params (role.id=030ee64221f74a029d408ae3bdb824a1, scope.project.id=c904620b9191403ca36d0aca020db817, user.id=bff19e37fa2243fda914f7f4e662c8f7) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:09.735640 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a6862-2634-4cc8-b61f-64c64881f5aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:09.737077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a6862-2634-4cc8-b61f-64c64881f5aa None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:09.743053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 639/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Jun 20 10:55:09 2022] GET /identity/v3/role_assignments?user.id=bff19e37fa2243fda914f7f4e662c8f7&scope.project.id=c904620b9191403ca36d0aca020db817&role.id=030ee64221f74a029d408ae3bdb824a1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:11.800316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20daec4-ae7a-4a6a-893b-67e62e056131 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:11.800925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20daec4-ae7a-4a6a-893b-67e62e056131 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:11.800925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20daec4-ae7a-4a6a-893b-67e62e056131 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:11.802314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 649/1288] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:55:11.808880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc050e-4143-4ab3-b450-dd9288cab1c8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:11.809696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc050e-4143-4ab3-b450-dd9288cab1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:11.809835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc050e-4143-4ab3-b450-dd9288cab1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:11.838553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-82fc050e-4143-4ab3-b450-dd9288cab1c8 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:55:11.839750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-82fc050e-4143-4ab3-b450-dd9288cab1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:55:11.874401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:11.915396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 640/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:55:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:55:11.938669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20daec4-ae7a-4a6a-893b-67e62e056131 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:12.063482 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20daec4-ae7a-4a6a-893b-67e62e056131 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:12.074123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:12.111568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d4813-a1e5-49de-9db6-139aaf4db177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:12.112377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d4813-a1e5-49de-9db6-139aaf4db177 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:12.112659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d4813-a1e5-49de-9db6-139aaf4db177 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:12.112932 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d4813-a1e5-49de-9db6-139aaf4db177 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:12.113593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d4813-a1e5-49de-9db6-139aaf4db177 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:12.115017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d4813-a1e5-49de-9db6-139aaf4db177 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e53d4813-a1e5-49de-9db6-139aaf4db177 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:12.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:12.129099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:12.130166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 650/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jun 20 10:55:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:12.135416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc050e-4143-4ab3-b450-dd9288cab1c8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:12.241028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc050e-4143-4ab3-b450-dd9288cab1c8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:12.276911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d065f09-86a2-4354-a43e-9b18b2126019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:12.277563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d065f09-86a2-4354-a43e-9b18b2126019 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:12.277842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d065f09-86a2-4354-a43e-9b18b2126019 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:12.278130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d065f09-86a2-4354-a43e-9b18b2126019 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:12.278842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d065f09-86a2-4354-a43e-9b18b2126019 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:12.279144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d065f09-86a2-4354-a43e-9b18b2126019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:12.280520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d065f09-86a2-4354-a43e-9b18b2126019 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:12.285597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 641/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jun 20 10:55:12 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:12.292594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d4813-a1e5-49de-9db6-139aaf4db177 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:12.408469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d4813-a1e5-49de-9db6-139aaf4db177 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:12.445512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa5661-ed8d-4a9b-b4fb-21638f3e02ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:12.446401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa5661-ed8d-4a9b-b4fb-21638f3e02ac None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:12.446672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa5661-ed8d-4a9b-b4fb-21638f3e02ac None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:12.446948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa5661-ed8d-4a9b-b4fb-21638f3e02ac None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:12.447657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa5661-ed8d-4a9b-b4fb-21638f3e02ac None admin] RBAC: Adding query filter params (type=placement) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:12.447964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa5661-ed8d-4a9b-b4fb-21638f3e02ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:12.449337 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa5661-ed8d-4a9b-b4fb-21638f3e02ac None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:12.455507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 651/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jun 20 10:55:12 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:14.442516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af2a32-e39e-4598-8d7c-8cee08374ab4 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:14.443089 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af2a32-e39e-4598-8d7c-8cee08374ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:14.443089 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af2a32-e39e-4598-8d7c-8cee08374ab4 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:14.445232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 642/1293] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:55:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:55:14.451525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a90a80-ece2-4c49-81ce-827a94f5d097 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:14.452000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a90a80-ece2-4c49-81ce-827a94f5d097 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:14.452102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a90a80-ece2-4c49-81ce-827a94f5d097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:14.473718 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f4a90a80-ece2-4c49-81ce-827a94f5d097 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:55:14.475741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f4a90a80-ece2-4c49-81ce-827a94f5d097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:55:14.507961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:14.547687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 652/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:55:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:55:14.571056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af2a32-e39e-4598-8d7c-8cee08374ab4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:14.610643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:14.709166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af2a32-e39e-4598-8d7c-8cee08374ab4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:14.749979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722b9b0-39eb-493d-a385-896c10edc031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:14.750685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722b9b0-39eb-493d-a385-896c10edc031 None admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:14.750969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722b9b0-39eb-493d-a385-896c10edc031 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:14.751244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722b9b0-39eb-493d-a385-896c10edc031 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:14.751969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722b9b0-39eb-493d-a385-896c10edc031 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:14.753370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722b9b0-39eb-493d-a385-896c10edc031 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:14.768028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 643/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Jun 20 10:55:14 2022] POST /identity/v3/services => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:55:16.791843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0580f58-358c-474a-abe9-1559e10e949a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:16.792660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0580f58-358c-474a-abe9-1559e10e949a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:16.792784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0580f58-358c-474a-abe9-1559e10e949a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:16.795018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 653/1296] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:55:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:55:16.801160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3487-b711-4d78-9018-d11d07ff8ad7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:16.801593 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3487-b711-4d78-9018-d11d07ff8ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:16.801693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3487-b711-4d78-9018-d11d07ff8ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:16.821962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b32f3487-b711-4d78-9018-d11d07ff8ad7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:55:16.823174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b32f3487-b711-4d78-9018-d11d07ff8ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:55:16.899063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 644/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:55:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 10:55:16.922832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0580f58-358c-474a-abe9-1559e10e949a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:17.031578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0580f58-358c-474a-abe9-1559e10e949a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:17.068654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03168a67-106c-4c0a-8682-9aa3a12538fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:17.069377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168a67-106c-4c0a-8682-9aa3a12538fd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:17.069657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168a67-106c-4c0a-8682-9aa3a12538fd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:17.069927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168a67-106c-4c0a-8682-9aa3a12538fd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:17.070625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03168a67-106c-4c0a-8682-9aa3a12538fd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:17.072050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03168a67-106c-4c0a-8682-9aa3a12538fd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-03168a67-106c-4c0a-8682-9aa3a12538fd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:17.084355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:17.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:17.086555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 654/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jun 20 10:55:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:17.092532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3487-b711-4d78-9018-d11d07ff8ad7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:17.112023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:17.198050 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3487-b711-4d78-9018-d11d07ff8ad7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:17.234880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc8dc-6072-4ab5-b034-2263d0f4d28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:17.235558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc8dc-6072-4ab5-b034-2263d0f4d28a None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:17.235839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc8dc-6072-4ab5-b034-2263d0f4d28a None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:17.236106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbc8dc-6072-4ab5-b034-2263d0f4d28a None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:17.236811 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbc8dc-6072-4ab5-b034-2263d0f4d28a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:17.237115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbc8dc-6072-4ab5-b034-2263d0f4d28a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:17.238523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbc8dc-6072-4ab5-b034-2263d0f4d28a None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:17.244198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 645/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jun 20 10:55:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:17.251169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03168a67-106c-4c0a-8682-9aa3a12538fd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:17.287998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:17.373002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03168a67-106c-4c0a-8682-9aa3a12538fd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:17.410394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca6c39-afb1-450d-82fa-dae0ef495e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:17.411060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca6c39-afb1-450d-82fa-dae0ef495e18 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:17.411348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca6c39-afb1-450d-82fa-dae0ef495e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:17.411628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca6c39-afb1-450d-82fa-dae0ef495e18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:17.412426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca6c39-afb1-450d-82fa-dae0ef495e18 None admin] RBAC: Adding query filter params (interface=public, service_id=c3dde524a4ae4148a3505cb5e9255662, region_id=RegionOne) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:17.412743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca6c39-afb1-450d-82fa-dae0ef495e18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:17.414153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca6c39-afb1-450d-82fa-dae0ef495e18 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:17.419658 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 655/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Jun 20 10:55:17 2022] GET /identity/v3/endpoints?service_id=c3dde524a4ae4148a3505cb5e9255662&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:17.428589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc8dc-6072-4ab5-b034-2263d0f4d28a None admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:17.522396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:17.541911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbc8dc-6072-4ab5-b034-2263d0f4d28a None admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:17.577065 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e3f38-353e-4548-9044-2f3f3e96b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:17.577719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e3f38-353e-4548-9044-2f3f3e96b212 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:17.578403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e3f38-353e-4548-9044-2f3f3e96b212 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:17.578684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e3f38-353e-4548-9044-2f3f3e96b212 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:17.579376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e3f38-353e-4548-9044-2f3f3e96b212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:17.580765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e3f38-353e-4548-9044-2f3f3e96b212 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:17.587880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 646/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Jun 20 10:55:17 2022] GET /identity/v3/services => generated 2191 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 10:55:19.608516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c162-2ebf-4c5a-8eb8-16bc2a94ae67 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:19.609347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c162-2ebf-4c5a-8eb8-16bc2a94ae67 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:19.609347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c162-2ebf-4c5a-8eb8-16bc2a94ae67 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:19.611590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 656/1302] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:55:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:55:19.619559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e3c32-4042-4d12-8ab2-920a4086a0b1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:19.620008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e3c32-4042-4d12-8ab2-920a4086a0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:19.620113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e3c32-4042-4d12-8ab2-920a4086a0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:19.638596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d88e3c32-4042-4d12-8ab2-920a4086a0b1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:55:19.639843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d88e3c32-4042-4d12-8ab2-920a4086a0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:55:19.712256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 647/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:55:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 10:55:19.741411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2c162-2ebf-4c5a-8eb8-16bc2a94ae67 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:19.844282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:19.851556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2c162-2ebf-4c5a-8eb8-16bc2a94ae67 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:19.886887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40b46d-3c5e-4ffd-9f8a-1700f5dd00df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:19.887605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40b46d-3c5e-4ffd-9f8a-1700f5dd00df None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:19.887883 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40b46d-3c5e-4ffd-9f8a-1700f5dd00df None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:19.888159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40b46d-3c5e-4ffd-9f8a-1700f5dd00df None admin] PATH_INFO: `/v3/services/placement` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:19.888804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40b46d-3c5e-4ffd-9f8a-1700f5dd00df None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:19.890199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40b46d-3c5e-4ffd-9f8a-1700f5dd00df None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6f40b46d-3c5e-4ffd-9f8a-1700f5dd00df None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 10:55:19.902081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 10:55:19.903124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Jun 20 10:55:19.903969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 657/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Jun 20 10:55:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 10:55:19.909415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e3c32-4042-4d12-8ab2-920a4086a0b1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:19.997738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:20.025960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e3c32-4042-4d12-8ab2-920a4086a0b1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:20.060918 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce853c1-dee6-4dfa-a289-ba84347042e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:20.061577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce853c1-dee6-4dfa-a289-ba84347042e7 None admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:20.061834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce853c1-dee6-4dfa-a289-ba84347042e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:20.062130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce853c1-dee6-4dfa-a289-ba84347042e7 None admin] PATH_INFO: `/v3/services` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:20.062860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce853c1-dee6-4dfa-a289-ba84347042e7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 10:55:20.063164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce853c1-dee6-4dfa-a289-ba84347042e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:20.064556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce853c1-dee6-4dfa-a289-ba84347042e7 None admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:20.069866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 648/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Jun 20 10:55:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 10:55:20.076325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40b46d-3c5e-4ffd-9f8a-1700f5dd00df None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:55:20.187011 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40b46d-3c5e-4ffd-9f8a-1700f5dd00df None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:55:20.221992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5df780-5ca7-40b8-8135-39b06cd38893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:55:20.222663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5df780-5ca7-40b8-8135-39b06cd38893 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:55:20.222941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5df780-5ca7-40b8-8135-39b06cd38893 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:55:20.223214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5df780-5ca7-40b8-8135-39b06cd38893 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:55:20.223942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5df780-5ca7-40b8-8135-39b06cd38893 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:55:20.225309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5df780-5ca7-40b8-8135-39b06cd38893 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:55:20.247038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 658/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Jun 20 10:55:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 10:55:30.066847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=30 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:55:30.242492 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:23.423244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f81bba-f7ec-440c-b612-19627b882037 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:23.424142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f81bba-f7ec-440c-b612-19627b882037 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:23.424566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f81bba-f7ec-440c-b612-19627b882037 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:23.426124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 649/1307] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:23.431549 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d5ceb-702a-438a-916a-f38687d20831 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:23.432336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d5ceb-702a-438a-916a-f38687d20831 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:23.432764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d5ceb-702a-438a-916a-f38687d20831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:23.534236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8c8d5ceb-702a-438a-916a-f38687d20831 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:23.535466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8c8d5ceb-702a-438a-916a-f38687d20831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:23.686183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 659/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:23.727680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c152-ed0c-4d93-bdff-9435cff2aa32 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:23.728053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c152-ed0c-4d93-bdff-9435cff2aa32 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:23.728466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c152-ed0c-4d93-bdff-9435cff2aa32 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:23.730062 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 650/1309] 10.222.0.33 () {54 vars in 964 bytes} [Mon Jun 20 10:56:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:23.745155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e42b20-9964-4476-b45d-8fa4f89f8d8d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:23.745490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e42b20-9964-4476-b45d-8fa4f89f8d8d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:23.745727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e42b20-9964-4476-b45d-8fa4f89f8d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:23.835421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-05e42b20-9964-4476-b45d-8fa4f89f8d8d None None] MFA Rules not processed for user `d73fe529c7d342f895aabcb9a39aba78`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:23.836640 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-05e42b20-9964-4476-b45d-8fa4f89f8d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:23.860080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:23.926477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 660/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Jun 20 10:56:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:23.935662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6c152-ed0c-4d93-bdff-9435cff2aa32 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:23.937501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3ba6c152-ed0c-4d93-bdff-9435cff2aa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:24.052625 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6c152-ed0c-4d93-bdff-9435cff2aa32 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:24.130851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e476729-1010-4793-82ce-680bd343d832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:24.131504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e476729-1010-4793-82ce-680bd343d832 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:24.131760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e476729-1010-4793-82ce-680bd343d832 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:24.132059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e476729-1010-4793-82ce-680bd343d832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:24.133681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3e476729-1010-4793-82ce-680bd343d832 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:24.201938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:24.270592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e476729-1010-4793-82ce-680bd343d832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:24.272219 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e476729-1010-4793-82ce-680bd343d832 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:24.291899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 651/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 10:56:26.589624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd166bf-033d-4d60-9a1e-72bfb99c9b0f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:26.590213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd166bf-033d-4d60-9a1e-72bfb99c9b0f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:26.590213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd166bf-033d-4d60-9a1e-72bfb99c9b0f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:26.591785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 661/1312] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:26.597032 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca14684-ca14-45b4-a19c-4bc337d61d75 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:26.597477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca14684-ca14-45b4-a19c-4bc337d61d75 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:26.597655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca14684-ca14-45b4-a19c-4bc337d61d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:26.677468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3ca14684-ca14-45b4-a19c-4bc337d61d75 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:26.678730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3ca14684-ca14-45b4-a19c-4bc337d61d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:26.798123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 652/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:26.836427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd166bf-033d-4d60-9a1e-72bfb99c9b0f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:26.838333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bfd166bf-033d-4d60-9a1e-72bfb99c9b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:26.951166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd166bf-033d-4d60-9a1e-72bfb99c9b0f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:27.031221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:27.034504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882056a-59e8-4d03-8a63-5fa79e1b4bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:27.035449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882056a-59e8-4d03-8a63-5fa79e1b4bbe service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:27.035767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882056a-59e8-4d03-8a63-5fa79e1b4bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:27.036446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882056a-59e8-4d03-8a63-5fa79e1b4bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:27.038684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f882056a-59e8-4d03-8a63-5fa79e1b4bbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:27.154307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882056a-59e8-4d03-8a63-5fa79e1b4bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:27.155779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882056a-59e8-4d03-8a63-5fa79e1b4bbe service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:27.177603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 662/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:56:29.516650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1db15-2e44-4afe-b621-8e9631bbf4eb None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:29.517181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1db15-2e44-4afe-b621-8e9631bbf4eb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:29.517181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1db15-2e44-4afe-b621-8e9631bbf4eb None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:29.518545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 653/1315] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:29.525079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6497-5494-44b2-bef0-b36d117bd4a9 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:29.525750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6497-5494-44b2-bef0-b36d117bd4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:29.525892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6497-5494-44b2-bef0-b36d117bd4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:29.603628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-464b6497-5494-44b2-bef0-b36d117bd4a9 None None] MFA Rules not processed for user `6fdb844dddc042b9958d0532be8d63c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:29.604850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-464b6497-5494-44b2-bef0-b36d117bd4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:29.659224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:29.705427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 663/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:29.762444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c502874-faa5-45b9-9bf6-760f5108983e None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:29.762750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c502874-faa5-45b9-9bf6-760f5108983e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:29.763130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c502874-faa5-45b9-9bf6-760f5108983e None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:29.765386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 654/1317] 10.222.0.33 () {54 vars in 964 bytes} [Mon Jun 20 10:56:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:29.772434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e2130-2789-4560-9d2c-20dc892ee27a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:29.772726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e2130-2789-4560-9d2c-20dc892ee27a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:29.773059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e2130-2789-4560-9d2c-20dc892ee27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:29.854976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5b7e2130-2789-4560-9d2c-20dc892ee27a None None] MFA Rules not processed for user `d73fe529c7d342f895aabcb9a39aba78`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:29.856187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5b7e2130-2789-4560-9d2c-20dc892ee27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:29.934939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 664/1318] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Jun 20 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:29.944219 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c502874-faa5-45b9-9bf6-760f5108983e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:29.945758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9c502874-faa5-45b9-9bf6-760f5108983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:29.956622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:30.063819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c502874-faa5-45b9-9bf6-760f5108983e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:30.137958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f110b7-2a66-47a5-ada9-d8ae82d92c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:30.138643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f110b7-2a66-47a5-ada9-d8ae82d92c55 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:30.138924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f110b7-2a66-47a5-ada9-d8ae82d92c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:30.139202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f110b7-2a66-47a5-ada9-d8ae82d92c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:30.140848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-37f110b7-2a66-47a5-ada9-d8ae82d92c55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:30.208003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:30.257635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f110b7-2a66-47a5-ada9-d8ae82d92c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:30.259190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f110b7-2a66-47a5-ada9-d8ae82d92c55 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:30.283112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 655/1319] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:56:33.242138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4c652-3a72-48a3-886e-913adf37e93c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:33.242695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4c652-3a72-48a3-886e-913adf37e93c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:33.242695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4c652-3a72-48a3-886e-913adf37e93c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:33.244007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 665/1320] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:33.250457 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec4440-7e75-4386-9a25-53a5ed90902e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:33.251139 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec4440-7e75-4386-9a25-53a5ed90902e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:33.251445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec4440-7e75-4386-9a25-53a5ed90902e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:33.324589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f2ec4440-7e75-4386-9a25-53a5ed90902e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:33.325845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f2ec4440-7e75-4386-9a25-53a5ed90902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:33.434960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 656/1321] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:33.477377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4c652-3a72-48a3-886e-913adf37e93c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:33.479027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5aa4c652-3a72-48a3-886e-913adf37e93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:33.532691 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:33.587897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4c652-3a72-48a3-886e-913adf37e93c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:33.664963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d7db8-92f1-4d13-85b5-fed443d3e4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:33.665615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d7db8-92f1-4d13-85b5-fed443d3e4fd service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:33.665896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d7db8-92f1-4d13-85b5-fed443d3e4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:33.666223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d7db8-92f1-4d13-85b5-fed443d3e4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:33.668375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-128d7db8-92f1-4d13-85b5-fed443d3e4fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:33.777646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:33.792593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d7db8-92f1-4d13-85b5-fed443d3e4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:33.793991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d7db8-92f1-4d13-85b5-fed443d3e4fd service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:33.812288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 666/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 10:56:36.560330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6335a-8be4-49d2-a0b7-a46832f7eaa3 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:36.561978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6335a-8be4-49d2-a0b7-a46832f7eaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:36.562407 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6335a-8be4-49d2-a0b7-a46832f7eaa3 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:36.565587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 657/1323] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:36.571250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e27a68-c5e3-4709-9061-1d4734d33ba1 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:36.571841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e27a68-c5e3-4709-9061-1d4734d33ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:36.572119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e27a68-c5e3-4709-9061-1d4734d33ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:36.637596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-99e27a68-c5e3-4709-9061-1d4734d33ba1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:36.638835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-99e27a68-c5e3-4709-9061-1d4734d33ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:36.752350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 667/1324] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:36.789789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6335a-8be4-49d2-a0b7-a46832f7eaa3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:36.791416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-28c6335a-8be4-49d2-a0b7-a46832f7eaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:36.815397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:36.904848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6335a-8be4-49d2-a0b7-a46832f7eaa3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:36.980960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bb994-ba27-4cfa-ad9c-6ef3582c3e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:36.981701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bb994-ba27-4cfa-ad9c-6ef3582c3e8d service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:36.981987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bb994-ba27-4cfa-ad9c-6ef3582c3e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:36.982270 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bb994-ba27-4cfa-ad9c-6ef3582c3e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:36.983942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-748bb994-ba27-4cfa-ad9c-6ef3582c3e8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:37.057797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:37.099583 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bb994-ba27-4cfa-ad9c-6ef3582c3e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:37.101042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bb994-ba27-4cfa-ad9c-6ef3582c3e8d service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:37.120446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 658/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:56:39.199987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2f327-1d77-4bf0-be59-13f001acb5d8 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:39.200524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2f327-1d77-4bf0-be59-13f001acb5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:39.200524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b2f327-1d77-4bf0-be59-13f001acb5d8 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:39.203345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 668/1326] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:56:39.207743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec307cb0-134f-4e15-a761-663386dc9c8d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:39.208190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec307cb0-134f-4e15-a761-663386dc9c8d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:39.208293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec307cb0-134f-4e15-a761-663386dc9c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:39.276289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ec307cb0-134f-4e15-a761-663386dc9c8d None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:39.277542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ec307cb0-134f-4e15-a761-663386dc9c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:39.376167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 659/1327] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:39.412085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2f327-1d77-4bf0-be59-13f001acb5d8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:39.413670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-17b2f327-1d77-4bf0-be59-13f001acb5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:39.486303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:39.534494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b2f327-1d77-4bf0-be59-13f001acb5d8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:39.615523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be533d-dc13-4d02-9e94-b4f55ce7ef38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:39.616183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be533d-dc13-4d02-9e94-b4f55ce7ef38 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:39.616463 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be533d-dc13-4d02-9e94-b4f55ce7ef38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:39.616737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be533d-dc13-4d02-9e94-b4f55ce7ef38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:39.618419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b5be533d-dc13-4d02-9e94-b4f55ce7ef38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:39.728181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:39.731200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be533d-dc13-4d02-9e94-b4f55ce7ef38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:39.732809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be533d-dc13-4d02-9e94-b4f55ce7ef38 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:39.758295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 669/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:56:41.836291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bd86d-d7e5-4b62-96f6-2dd96ceb2a19 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:41.836872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bd86d-d7e5-4b62-96f6-2dd96ceb2a19 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:41.836872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bd86d-d7e5-4b62-96f6-2dd96ceb2a19 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:41.838295 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 660/1329] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:41.844388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722d347-18cb-47f1-aac6-53a28de078f7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:41.844965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722d347-18cb-47f1-aac6-53a28de078f7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:41.844965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722d347-18cb-47f1-aac6-53a28de078f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:41.912992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b722d347-18cb-47f1-aac6-53a28de078f7 None None] MFA Rules not processed for user `6fdb844dddc042b9958d0532be8d63c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:41.914271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b722d347-18cb-47f1-aac6-53a28de078f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:42.022645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 670/1330] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:42.060627 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bd86d-d7e5-4b62-96f6-2dd96ceb2a19 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:42.062178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-384bd86d-d7e5-4b62-96f6-2dd96ceb2a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:42.095229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:42.170059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bd86d-d7e5-4b62-96f6-2dd96ceb2a19 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:42.245803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eab7d0-c187-45d1-a139-7aa1c52edb91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:42.246488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eab7d0-c187-45d1-a139-7aa1c52edb91 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:42.246769 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eab7d0-c187-45d1-a139-7aa1c52edb91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:42.247045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eab7d0-c187-45d1-a139-7aa1c52edb91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:42.248683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d1eab7d0-c187-45d1-a139-7aa1c52edb91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:42.343570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:42.379788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eab7d0-c187-45d1-a139-7aa1c52edb91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:42.381261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eab7d0-c187-45d1-a139-7aa1c52edb91 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:42.401873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 661/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:56:44.722508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e08a9-d27e-4d8b-9e2c-1a563361b7d5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:44.723060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e08a9-d27e-4d8b-9e2c-1a563361b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:44.723060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e08a9-d27e-4d8b-9e2c-1a563361b7d5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:44.724378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 671/1332] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:44.731001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ad79f-c5e2-4842-bd16-d9f176920ffb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:44.731719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ad79f-c5e2-4842-bd16-d9f176920ffb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:44.731869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3ad79f-c5e2-4842-bd16-d9f176920ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:44.806076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0b3ad79f-c5e2-4842-bd16-d9f176920ffb None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:44.807425 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0b3ad79f-c5e2-4842-bd16-d9f176920ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:44.917481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 662/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:44.957463 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e08a9-d27e-4d8b-9e2c-1a563361b7d5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:44.959048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-842e08a9-d27e-4d8b-9e2c-1a563361b7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:45.032793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:45.070443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e08a9-d27e-4d8b-9e2c-1a563361b7d5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:45.143634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819139b-523d-4326-8c2f-70133720dbf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:45.144294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819139b-523d-4326-8c2f-70133720dbf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:45.144581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819139b-523d-4326-8c2f-70133720dbf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:45.144857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819139b-523d-4326-8c2f-70133720dbf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:45.146615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5819139b-523d-4326-8c2f-70133720dbf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:45.259245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819139b-523d-4326-8c2f-70133720dbf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:45.260758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819139b-523d-4326-8c2f-70133720dbf4 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:45.270945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:45.280696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 672/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:56:47.326947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8d98-1a90-4cf3-817e-e46f94e11454 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:47.327836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8d98-1a90-4cf3-817e-e46f94e11454 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:47.327836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8d98-1a90-4cf3-817e-e46f94e11454 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:47.330014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 663/1335] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:47.336299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a57e-e19f-4045-a299-d1ca737a6ba1 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:47.336980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a57e-e19f-4045-a299-d1ca737a6ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:47.337093 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4a57e-e19f-4045-a299-d1ca737a6ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:47.411217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-49e4a57e-e19f-4045-a299-d1ca737a6ba1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:47.412446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-49e4a57e-e19f-4045-a299-d1ca737a6ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:47.519173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 673/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:47.565223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef8d98-1a90-4cf3-817e-e46f94e11454 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:47.567736 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b3ef8d98-1a90-4cf3-817e-e46f94e11454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:47.617255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:47.695613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef8d98-1a90-4cf3-817e-e46f94e11454 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:47.777060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575e744-7d0d-433b-b46e-1479d0b3b12a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:47.777708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575e744-7d0d-433b-b46e-1479d0b3b12a service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:47.777985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575e744-7d0d-433b-b46e-1479d0b3b12a service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:47.778262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575e744-7d0d-433b-b46e-1479d0b3b12a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:47.779914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f575e744-7d0d-433b-b46e-1479d0b3b12a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:47.869764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:47.896086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575e744-7d0d-433b-b46e-1479d0b3b12a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:47.897540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575e744-7d0d-433b-b46e-1479d0b3b12a service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:47.915944 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 664/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:56:50.305346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29823413-f711-4009-91fe-3e5a51162919 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:50.305925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29823413-f711-4009-91fe-3e5a51162919 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:50.305925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29823413-f711-4009-91fe-3e5a51162919 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:50.308095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 674/1338] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:50.313538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185ecb5-948f-4e77-8370-83aae4ec45be None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:50.314019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185ecb5-948f-4e77-8370-83aae4ec45be None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:50.314150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185ecb5-948f-4e77-8370-83aae4ec45be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:50.384123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f185ecb5-948f-4e77-8370-83aae4ec45be None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:50.385953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f185ecb5-948f-4e77-8370-83aae4ec45be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:50.486254 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 665/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:50.526564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29823413-f711-4009-91fe-3e5a51162919 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:50.528127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-29823413-f711-4009-91fe-3e5a51162919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:50.607063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:50.641293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29823413-f711-4009-91fe-3e5a51162919 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:50.726146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f59c8-d5b7-4ea5-8b60-08065542db08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:50.726821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f59c8-d5b7-4ea5-8b60-08065542db08 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:50.727103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f59c8-d5b7-4ea5-8b60-08065542db08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:50.727370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f59c8-d5b7-4ea5-8b60-08065542db08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:50.729171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-232f59c8-d5b7-4ea5-8b60-08065542db08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:50.850406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f59c8-d5b7-4ea5-8b60-08065542db08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:50.851866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f59c8-d5b7-4ea5-8b60-08065542db08 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:50.871573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 675/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 10:56:54.932204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40aeed-238c-4916-9aa7-b35fa35f3568 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:54.932756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40aeed-238c-4916-9aa7-b35fa35f3568 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:54.932756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40aeed-238c-4916-9aa7-b35fa35f3568 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:54.934024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 666/1341] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:54.939491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e4945-a90f-4601-a2c7-7f180f004c76 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:54.939917 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e4945-a90f-4601-a2c7-7f180f004c76 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:54.940029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e4945-a90f-4601-a2c7-7f180f004c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:54.946438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:55.014061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-be2e4945-a90f-4601-a2c7-7f180f004c76 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:55.015371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-be2e4945-a90f-4601-a2c7-7f180f004c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:55.124083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 676/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:55.162320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40aeed-238c-4916-9aa7-b35fa35f3568 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:55.164497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1f40aeed-238c-4916-9aa7-b35fa35f3568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:55.218308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:56:55.273733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40aeed-238c-4916-9aa7-b35fa35f3568 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:55.278387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd8dc-adb1-46e5-99c8-dcfb742ca6a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:55.279117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd8dc-adb1-46e5-99c8-dcfb742ca6a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:55.279456 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd8dc-adb1-46e5-99c8-dcfb742ca6a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:55.279797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd8dc-adb1-46e5-99c8-dcfb742ca6a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:55.289413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cd8dc-adb1-46e5-99c8-dcfb742ca6a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:55.291060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cd8dc-adb1-46e5-99c8-dcfb742ca6a5 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:55.301174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 667/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:56:57.967976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e528b-78fb-4f83-826e-c758d323b994 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:57.968572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e528b-78fb-4f83-826e-c758d323b994 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:57.968572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e528b-78fb-4f83-826e-c758d323b994 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:57.969948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 677/1344] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:56:57.975340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f83bff-65af-484b-a6c6-da1aa0c150fd None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:57.975821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f83bff-65af-484b-a6c6-da1aa0c150fd None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:57.975929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f83bff-65af-484b-a6c6-da1aa0c150fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:58.008079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-77f83bff-65af-484b-a6c6-da1aa0c150fd None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:56:58.009314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-77f83bff-65af-484b-a6c6-da1aa0c150fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:58.097181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 668/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:56:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:56:58.135306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e528b-78fb-4f83-826e-c758d323b994 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:56:58.137309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-234e528b-78fb-4f83-826e-c758d323b994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:56:58.152608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e528b-78fb-4f83-826e-c758d323b994 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:56:58.156714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490d9d8-cfa8-4509-8d06-1bdbad4126d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:56:58.157376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490d9d8-cfa8-4509-8d06-1bdbad4126d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:56:58.157709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490d9d8-cfa8-4509-8d06-1bdbad4126d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:56:58.157938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490d9d8-cfa8-4509-8d06-1bdbad4126d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:56:58.168412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490d9d8-cfa8-4509-8d06-1bdbad4126d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:56:58.169896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490d9d8-cfa8-4509-8d06-1bdbad4126d6 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:56:58.180951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 678/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:56:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 10:57:02.546477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560cc52-4068-454d-9f81-6f75b3531dc2 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:02.547228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560cc52-4068-454d-9f81-6f75b3531dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:02.547952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560cc52-4068-454d-9f81-6f75b3531dc2 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:02.549453 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 669/1347] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:02.554853 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bba80-0e12-4465-baeb-b43b4316be8b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:02.555352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bba80-0e12-4465-baeb-b43b4316be8b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:02.555438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bba80-0e12-4465-baeb-b43b4316be8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:02.573640 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-301bba80-0e12-4465-baeb-b43b4316be8b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:02.574879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-301bba80-0e12-4465-baeb-b43b4316be8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:02.583606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 679/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:02.621329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560cc52-4068-454d-9f81-6f75b3531dc2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:02.633557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560cc52-4068-454d-9f81-6f75b3531dc2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:02.637470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d0cfb-aa51-46e6-972c-8ef36fc0a4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:02.638147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d0cfb-aa51-46e6-972c-8ef36fc0a4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:02.638431 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d0cfb-aa51-46e6-972c-8ef36fc0a4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:02.638709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d0cfb-aa51-46e6-972c-8ef36fc0a4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:02.649248 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d0cfb-aa51-46e6-972c-8ef36fc0a4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:02.651109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d0cfb-aa51-46e6-972c-8ef36fc0a4f3 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:02.661403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 670/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:07.764583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185c423-f84f-4a5f-8a05-06ff2e562996 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:07.765681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185c423-f84f-4a5f-8a05-06ff2e562996 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:07.766197 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185c423-f84f-4a5f-8a05-06ff2e562996 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:07.768113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 680/1350] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 10:57:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:07.773054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc68134-ffd1-4237-96c7-3d6cc7e1c765 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:07.773505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc68134-ffd1-4237-96c7-3d6cc7e1c765 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:07.773606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc68134-ffd1-4237-96c7-3d6cc7e1c765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:07.845767 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-acc68134-ffd1-4237-96c7-3d6cc7e1c765 None None] MFA Rules not processed for user `bff19e37fa2243fda914f7f4e662c8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:07.847103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-acc68134-ffd1-4237-96c7-3d6cc7e1c765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:07.875382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:57:07.947302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 671/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 10:57:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 10:57:12.117467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d7d9e-772f-4c79-951f-c9adaae39f09 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:12.118033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d7d9e-772f-4c79-951f-c9adaae39f09 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:12.118033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d7d9e-772f-4c79-951f-c9adaae39f09 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:12.119796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 681/1352] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 10:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:12.125241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26d4f6-36e2-4e1b-9f41-8ace1423f1df None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:12.125675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26d4f6-36e2-4e1b-9f41-8ace1423f1df None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:12.125797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26d4f6-36e2-4e1b-9f41-8ace1423f1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:12.145651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1b26d4f6-36e2-4e1b-9f41-8ace1423f1df None None] MFA Rules not processed for user `bff19e37fa2243fda914f7f4e662c8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:12.147229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1b26d4f6-36e2-4e1b-9f41-8ace1423f1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:12.157088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 672/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 10:57:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:12.568307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=72 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:57:13.673509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71740fe-689c-488a-b161-c9bbb41511b0 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:13.674186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71740fe-689c-488a-b161-c9bbb41511b0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:13.674186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71740fe-689c-488a-b161-c9bbb41511b0 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:13.675779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 682/1354] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 10:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:13.681575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f60c-21f5-40ab-b19d-59d97287c20e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:13.682014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f60c-21f5-40ab-b19d-59d97287c20e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:13.682202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f60c-21f5-40ab-b19d-59d97287c20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:13.701246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4974f60c-21f5-40ab-b19d-59d97287c20e None None] MFA Rules not processed for user `bff19e37fa2243fda914f7f4e662c8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:13.702479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4974f60c-21f5-40ab-b19d-59d97287c20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:13.711115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 673/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 10:57:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:17.340714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3593-6e95-4f88-86ac-fd8b172d550f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:17.341654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3593-6e95-4f88-86ac-fd8b172d550f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:17.342208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3593-6e95-4f88-86ac-fd8b172d550f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:17.344134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 683/1356] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:17.350030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e84f1-b4aa-48e2-9a2b-7a55ab6d78d2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:17.350979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e84f1-b4aa-48e2-9a2b-7a55ab6d78d2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:17.351481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e84f1-b4aa-48e2-9a2b-7a55ab6d78d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:17.371127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-639e84f1-b4aa-48e2-9a2b-7a55ab6d78d2 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:17.372315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-639e84f1-b4aa-48e2-9a2b-7a55ab6d78d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:17.380731 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 674/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:17.436099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735daa2-d078-4219-9652-622c027067b0 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:17.436650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735daa2-d078-4219-9652-622c027067b0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:17.437163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735daa2-d078-4219-9652-622c027067b0 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:17.439351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 684/1358] 10.222.0.33 () {54 vars in 961 bytes} [Mon Jun 20 10:57:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:17.448841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df004a-e20a-4f1e-9ed4-1c2f18f28d79 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:17.449656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df004a-e20a-4f1e-9ed4-1c2f18f28d79 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:17.450700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df004a-e20a-4f1e-9ed4-1c2f18f28d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:17.504007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-81df004a-e20a-4f1e-9ed4-1c2f18f28d79 None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:17.505289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-81df004a-e20a-4f1e-9ed4-1c2f18f28d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:17.595962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 675/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Jun 20 10:57:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:17.603973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735daa2-d078-4219-9652-622c027067b0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:17.620026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735daa2-d078-4219-9652-622c027067b0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:17.625944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dae455-d86a-448d-9f05-f8f76491811f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:17.626876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dae455-d86a-448d-9f05-f8f76491811f service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:17.627359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dae455-d86a-448d-9f05-f8f76491811f service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:17.627836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dae455-d86a-448d-9f05-f8f76491811f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:17.637730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dae455-d86a-448d-9f05-f8f76491811f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:17.640077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dae455-d86a-448d-9f05-f8f76491811f service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:17.651892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 685/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:19.925729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af520bbc-1501-421b-a82b-ccd4de051904 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:19.926336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af520bbc-1501-421b-a82b-ccd4de051904 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:19.926336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af520bbc-1501-421b-a82b-ccd4de051904 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:19.927908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 676/1361] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:19.933387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a92efd-350e-49b3-8277-2ac5d4a816ee None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:19.933839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a92efd-350e-49b3-8277-2ac5d4a816ee None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:19.933931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a92efd-350e-49b3-8277-2ac5d4a816ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:19.953048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-58a92efd-350e-49b3-8277-2ac5d4a816ee None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:19.954292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-58a92efd-350e-49b3-8277-2ac5d4a816ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:19.962901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 686/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:20.007629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af520bbc-1501-421b-a82b-ccd4de051904 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:20.022422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af520bbc-1501-421b-a82b-ccd4de051904 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:20.028561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfe7fe-0b3c-4f13-86f3-595248cfe180 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:20.029206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfe7fe-0b3c-4f13-86f3-595248cfe180 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:20.029499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfe7fe-0b3c-4f13-86f3-595248cfe180 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:20.029771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfe7fe-0b3c-4f13-86f3-595248cfe180 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:20.038830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfe7fe-0b3c-4f13-86f3-595248cfe180 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:20.040428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfe7fe-0b3c-4f13-86f3-595248cfe180 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:20.051405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 677/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:22.470939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d73e0-81a0-4549-b5d1-cc4b47fd1e64 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:22.471411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d73e0-81a0-4549-b5d1-cc4b47fd1e64 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:22.471517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d73e0-81a0-4549-b5d1-cc4b47fd1e64 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:22.473142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 687/1364] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:57:22.479230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e0c0-1802-43d5-8836-3621ede7c701 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:22.479674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e0c0-1802-43d5-8836-3621ede7c701 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:22.479808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162e0c0-1802-43d5-8836-3621ede7c701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:22.501340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7162e0c0-1802-43d5-8836-3621ede7c701 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:22.502621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7162e0c0-1802-43d5-8836-3621ede7c701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:22.511549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 678/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:22.557570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89607960-ffd8-4f89-9d15-d44d3d1e183e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:22.557939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89607960-ffd8-4f89-9d15-d44d3d1e183e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:22.558308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89607960-ffd8-4f89-9d15-d44d3d1e183e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:22.559747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 688/1366] 10.222.0.33 () {54 vars in 961 bytes} [Mon Jun 20 10:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:22.565807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d4957-341a-4702-9a60-bfa3bd7167d9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:22.566227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d4957-341a-4702-9a60-bfa3bd7167d9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:22.566577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d4957-341a-4702-9a60-bfa3bd7167d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:22.595573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-111d4957-341a-4702-9a60-bfa3bd7167d9 None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:22.596776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-111d4957-341a-4702-9a60-bfa3bd7167d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:22.608231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 679/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Jun 20 10:57:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:22.616140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89607960-ffd8-4f89-9d15-d44d3d1e183e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:22.632405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89607960-ffd8-4f89-9d15-d44d3d1e183e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:22.643404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059ff05-b090-4cc8-a91c-f630a8a16985 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:22.644068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ff05-b090-4cc8-a91c-f630a8a16985 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:22.644371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ff05-b090-4cc8-a91c-f630a8a16985 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:22.644799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ff05-b090-4cc8-a91c-f630a8a16985 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:22.656350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059ff05-b090-4cc8-a91c-f630a8a16985 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:22.659055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059ff05-b090-4cc8-a91c-f630a8a16985 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:22.673255 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 689/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 10:57:24.794413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854320de-b4f2-44ec-aa8b-b3a3fad3b4d7 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:24.794970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854320de-b4f2-44ec-aa8b-b3a3fad3b4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:24.794970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854320de-b4f2-44ec-aa8b-b3a3fad3b4d7 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:24.796315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 680/1369] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:24.801702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670805ee-85ed-4ce3-9cac-c2d730944140 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:24.802174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670805ee-85ed-4ce3-9cac-c2d730944140 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:24.802260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670805ee-85ed-4ce3-9cac-c2d730944140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:24.824470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-670805ee-85ed-4ce3-9cac-c2d730944140 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:24.826638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-670805ee-85ed-4ce3-9cac-c2d730944140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:24.837416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 690/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:24.881520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854320de-b4f2-44ec-aa8b-b3a3fad3b4d7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:24.899301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854320de-b4f2-44ec-aa8b-b3a3fad3b4d7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:24.907948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b52e0a-30de-476c-8024-84a73d0488b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:24.908876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b52e0a-30de-476c-8024-84a73d0488b7 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:24.909851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b52e0a-30de-476c-8024-84a73d0488b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:24.910299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b52e0a-30de-476c-8024-84a73d0488b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:24.922452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b52e0a-30de-476c-8024-84a73d0488b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:24.924059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b52e0a-30de-476c-8024-84a73d0488b7 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:24.934122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 681/1371] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:25.292761 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37838ea4-5e2a-4c31-9606-a5f21a58704e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:25.293047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37838ea4-5e2a-4c31-9606-a5f21a58704e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:25.293544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37838ea4-5e2a-4c31-9606-a5f21a58704e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:25.298485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 691/1372] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:57:25.300651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3660fde-9861-4b35-bce9-a2aed5e7105b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:25.300942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3660fde-9861-4b35-bce9-a2aed5e7105b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:25.301482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3660fde-9861-4b35-bce9-a2aed5e7105b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:25.323260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c3660fde-9861-4b35-bce9-a2aed5e7105b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:25.324482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c3660fde-9861-4b35-bce9-a2aed5e7105b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:25.333626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 682/1373] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 10:57:25.681889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1794f36-42ea-4c25-b7e9-aab1b4dd955c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:25.681889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1794f36-42ea-4c25-b7e9-aab1b4dd955c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:25.681889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1794f36-42ea-4c25-b7e9-aab1b4dd955c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:25.684759 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 692/1374] 10.222.0.33 () {54 vars in 963 bytes} [Mon Jun 20 10:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:25.688819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8368a8-3d2a-4d6c-bd03-6d7fe583313d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:25.689154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8368a8-3d2a-4d6c-bd03-6d7fe583313d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:25.689340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8368a8-3d2a-4d6c-bd03-6d7fe583313d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:25.698259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:57:25.748985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ed8368a8-3d2a-4d6c-bd03-6d7fe583313d None None] MFA Rules not processed for user `841e89a6caef4a089edc0f3b1df2fe9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:25.750127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ed8368a8-3d2a-4d6c-bd03-6d7fe583313d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:25.814986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 683/1375] 10.222.0.33 () {58 vars in 1060 bytes} [Mon Jun 20 10:57:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:25.822140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1794f36-42ea-4c25-b7e9-aab1b4dd955c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:25.834269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1794f36-42ea-4c25-b7e9-aab1b4dd955c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:25.838198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e4bb6-77df-4672-9b66-758eaf6767ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:25.838897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e4bb6-77df-4672-9b66-758eaf6767ff service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:25.839181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e4bb6-77df-4672-9b66-758eaf6767ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:25.839571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e4bb6-77df-4672-9b66-758eaf6767ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:25.849180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e4bb6-77df-4672-9b66-758eaf6767ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:25.850772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e4bb6-77df-4672-9b66-758eaf6767ff service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:25.861113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 693/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:26.996803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663a46-cfed-4115-9949-faa4f48179a2 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:26.997353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663a46-cfed-4115-9949-faa4f48179a2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:26.997353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663a46-cfed-4115-9949-faa4f48179a2 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:26.998672 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 684/1377] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:27.004124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e992a-4249-4563-9058-6976f7c918f8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:27.004543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e992a-4249-4563-9058-6976f7c918f8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:27.004765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e992a-4249-4563-9058-6976f7c918f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:27.023700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-690e992a-4249-4563-9058-6976f7c918f8 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:27.024966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-690e992a-4249-4563-9058-6976f7c918f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:27.033988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 694/1378] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:27.079060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec663a46-cfed-4115-9949-faa4f48179a2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:27.094291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec663a46-cfed-4115-9949-faa4f48179a2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:27.100015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ff7a2-1029-4c13-bcab-f16778209939 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:27.100661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ff7a2-1029-4c13-bcab-f16778209939 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:27.101081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ff7a2-1029-4c13-bcab-f16778209939 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:27.101358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ff7a2-1029-4c13-bcab-f16778209939 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:27.110567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ff7a2-1029-4c13-bcab-f16778209939 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:27.112194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ff7a2-1029-4c13-bcab-f16778209939 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:27.122719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 685/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 10:57:29.204624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41aaab-374c-456b-af96-9b549ce396c5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:29.205144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41aaab-374c-456b-af96-9b549ce396c5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:29.205144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41aaab-374c-456b-af96-9b549ce396c5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:29.206499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 695/1380] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:29.212026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fc484-e74c-47b0-b120-4fae78f333da None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:29.212454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fc484-e74c-47b0-b120-4fae78f333da None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:29.212647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fc484-e74c-47b0-b120-4fae78f333da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:29.231293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0a2fc484-e74c-47b0-b120-4fae78f333da None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:29.232504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0a2fc484-e74c-47b0-b120-4fae78f333da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:29.245063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 686/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:29.294772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41aaab-374c-456b-af96-9b549ce396c5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:29.310112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41aaab-374c-456b-af96-9b549ce396c5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:29.315525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d509e-2256-414b-b3c7-2791f4ea2f45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:29.316166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d509e-2256-414b-b3c7-2791f4ea2f45 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:29.316447 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d509e-2256-414b-b3c7-2791f4ea2f45 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:29.316719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d509e-2256-414b-b3c7-2791f4ea2f45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:29.335345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d509e-2256-414b-b3c7-2791f4ea2f45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:29.338531 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d509e-2256-414b-b3c7-2791f4ea2f45 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:29.351968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 696/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:31.438521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9404b58-f13c-4726-b6a9-29af3e2e1bf9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:31.439117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9404b58-f13c-4726-b6a9-29af3e2e1bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:31.439117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9404b58-f13c-4726-b6a9-29af3e2e1bf9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:31.440436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 687/1383] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:31.446020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88361a-7031-4ae7-ac4a-556dfdb3276e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:31.446654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88361a-7031-4ae7-ac4a-556dfdb3276e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:31.446886 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88361a-7031-4ae7-ac4a-556dfdb3276e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:31.465963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cd88361a-7031-4ae7-ac4a-556dfdb3276e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:31.467298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cd88361a-7031-4ae7-ac4a-556dfdb3276e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:31.477357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 697/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 10:57:31.519648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9404b58-f13c-4726-b6a9-29af3e2e1bf9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:31.536393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9404b58-f13c-4726-b6a9-29af3e2e1bf9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:31.542667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28725e49-35ea-4ea1-b5ab-4666ce5f2850 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:31.543570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28725e49-35ea-4ea1-b5ab-4666ce5f2850 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:31.544147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28725e49-35ea-4ea1-b5ab-4666ce5f2850 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:31.544519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28725e49-35ea-4ea1-b5ab-4666ce5f2850 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:31.556878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28725e49-35ea-4ea1-b5ab-4666ce5f2850 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:31.558651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28725e49-35ea-4ea1-b5ab-4666ce5f2850 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:31.570853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 688/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:33.674917 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23962aa-a130-4a21-8443-bc63f179339b None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:33.675498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23962aa-a130-4a21-8443-bc63f179339b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:33.675498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23962aa-a130-4a21-8443-bc63f179339b None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:33.677584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 698/1386] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:33.684523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc43dfd-0df4-44be-aa86-9ccd1ce4568d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:33.685017 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc43dfd-0df4-44be-aa86-9ccd1ce4568d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:33.685145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc43dfd-0df4-44be-aa86-9ccd1ce4568d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:33.708161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4dc43dfd-0df4-44be-aa86-9ccd1ce4568d None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:33.709847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4dc43dfd-0df4-44be-aa86-9ccd1ce4568d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:33.720880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 689/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:33.787678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23962aa-a130-4a21-8443-bc63f179339b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:33.805919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23962aa-a130-4a21-8443-bc63f179339b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:33.816966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd69754-112d-42c6-8989-8ad5148d0b96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:33.817690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd69754-112d-42c6-8989-8ad5148d0b96 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:33.818010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd69754-112d-42c6-8989-8ad5148d0b96 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:33.818335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd69754-112d-42c6-8989-8ad5148d0b96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:33.830730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd69754-112d-42c6-8989-8ad5148d0b96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:33.832452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd69754-112d-42c6-8989-8ad5148d0b96 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:33.843012 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 699/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:35.942400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0f13e-b3d8-4df1-8747-df43793ee8f5 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:35.943095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0f13e-b3d8-4df1-8747-df43793ee8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:35.943095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0f13e-b3d8-4df1-8747-df43793ee8f5 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:35.944451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 690/1389] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:35.951308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fd493-97f2-4d9d-a150-6a44434eeca3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:35.951850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fd493-97f2-4d9d-a150-6a44434eeca3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:35.951850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fd493-97f2-4d9d-a150-6a44434eeca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:35.972761 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c50fd493-97f2-4d9d-a150-6a44434eeca3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:35.974054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c50fd493-97f2-4d9d-a150-6a44434eeca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:35.983176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 700/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 10:57:36.035332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0f13e-b3d8-4df1-8747-df43793ee8f5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:36.050552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0f13e-b3d8-4df1-8747-df43793ee8f5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:36.056119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d51615-9419-4303-8f17-25b5c61223fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:36.056783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d51615-9419-4303-8f17-25b5c61223fd service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:36.057071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d51615-9419-4303-8f17-25b5c61223fd service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:36.057351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d51615-9419-4303-8f17-25b5c61223fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:36.070944 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d51615-9419-4303-8f17-25b5c61223fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:36.072893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d51615-9419-4303-8f17-25b5c61223fd service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:36.085961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 691/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 10:57:38.329566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194d9ed-356c-497e-899f-da2f3f9b6d76 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:38.330634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194d9ed-356c-497e-899f-da2f3f9b6d76 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:38.331010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194d9ed-356c-497e-899f-da2f3f9b6d76 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:38.333171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 701/1392] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:38.339586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ec71d-b592-4a81-99e1-9f34e1550100 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:38.340074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ec71d-b592-4a81-99e1-9f34e1550100 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:38.340331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ec71d-b592-4a81-99e1-9f34e1550100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:38.365197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7f4ec71d-b592-4a81-99e1-9f34e1550100 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:38.367087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7f4ec71d-b592-4a81-99e1-9f34e1550100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:38.378017 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 692/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:38.436934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194d9ed-356c-497e-899f-da2f3f9b6d76 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:38.458384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194d9ed-356c-497e-899f-da2f3f9b6d76 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:38.466348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fc231-43fd-40ad-ae2d-9a0bf27099e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:38.467331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fc231-43fd-40ad-ae2d-9a0bf27099e6 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:38.467708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fc231-43fd-40ad-ae2d-9a0bf27099e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:38.468072 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fc231-43fd-40ad-ae2d-9a0bf27099e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:38.480049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fc231-43fd-40ad-ae2d-9a0bf27099e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:38.482323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fc231-43fd-40ad-ae2d-9a0bf27099e6 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:38.496942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 702/1394] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:40.572855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc834c6-4c10-49a1-998e-7081dc9a449f None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:40.573429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc834c6-4c10-49a1-998e-7081dc9a449f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:40.573429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc834c6-4c10-49a1-998e-7081dc9a449f None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:40.575238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 693/1395] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:40.580765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc6c2a-d2e6-4818-adcb-b7d609f6ca3e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:40.581188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc6c2a-d2e6-4818-adcb-b7d609f6ca3e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:40.581346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc6c2a-d2e6-4818-adcb-b7d609f6ca3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:40.600846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c0fc6c2a-d2e6-4818-adcb-b7d609f6ca3e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:40.602040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c0fc6c2a-d2e6-4818-adcb-b7d609f6ca3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:40.611655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 703/1396] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:40.656383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc834c6-4c10-49a1-998e-7081dc9a449f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:40.671989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc834c6-4c10-49a1-998e-7081dc9a449f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:40.677817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cf2fc-bc61-4daa-8740-d4074a6c50ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:40.678524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf2fc-bc61-4daa-8740-d4074a6c50ac service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:40.678809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf2fc-bc61-4daa-8740-d4074a6c50ac service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:40.679082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cf2fc-bc61-4daa-8740-d4074a6c50ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:40.687979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cf2fc-bc61-4daa-8740-d4074a6c50ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:40.689883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903cf2fc-bc61-4daa-8740-d4074a6c50ac service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:40.700130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 694/1397] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:41.005779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88efb52-f2c9-4df4-96f1-458a9b3e3a0f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:41.006176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88efb52-f2c9-4df4-96f1-458a9b3e3a0f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:41.006450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88efb52-f2c9-4df4-96f1-458a9b3e3a0f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:41.007891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 704/1398] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:41.013605 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ba14-d8c6-4529-84a4-39ac7f419103 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:41.013977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ba14-d8c6-4529-84a4-39ac7f419103 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:41.014362 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196ba14-d8c6-4529-84a4-39ac7f419103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:41.033630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b196ba14-d8c6-4529-84a4-39ac7f419103 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:41.034853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b196ba14-d8c6-4529-84a4-39ac7f419103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:41.043947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 695/1399] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 10:57:41.100259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1ac9d-2a1c-4ab9-91db-ff4f43ab6754 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:41.100647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1ac9d-2a1c-4ab9-91db-ff4f43ab6754 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:41.100994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd1ac9d-2a1c-4ab9-91db-ff4f43ab6754 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:41.102525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 705/1400] 10.222.0.33 () {54 vars in 962 bytes} [Mon Jun 20 10:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:41.107976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51870fa-cc6a-4c2b-b4f1-276806448d81 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:41.108272 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51870fa-cc6a-4c2b-b4f1-276806448d81 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:41.108553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51870fa-cc6a-4c2b-b4f1-276806448d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:41.160716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b51870fa-cc6a-4c2b-b4f1-276806448d81 None None] MFA Rules not processed for user `e4bf4d9571d541bcbba87094198dc2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:41.163279 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b51870fa-cc6a-4c2b-b4f1-276806448d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:41.203437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:57:41.234640 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 696/1401] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Jun 20 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:41.242502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1ac9d-2a1c-4ab9-91db-ff4f43ab6754 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:41.254001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd1ac9d-2a1c-4ab9-91db-ff4f43ab6754 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:41.258038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f17b7-1694-4658-acba-059c5bb71cb1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bf4d9571d541bcbba87094198dc2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:41.259019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f17b7-1694-4658-acba-059c5bb71cb1 service glance] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:41.259484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f17b7-1694-4658-acba-059c5bb71cb1 service glance] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:41.259747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f17b7-1694-4658-acba-059c5bb71cb1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:41.269675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f17b7-1694-4658-acba-059c5bb71cb1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:41.271155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f17b7-1694-4658-acba-059c5bb71cb1 service glance] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:41.281145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 706/1402] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:42.004160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5206919-2f96-404f-b289-d78013e3112e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:42.004677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5206919-2f96-404f-b289-d78013e3112e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:42.004677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5206919-2f96-404f-b289-d78013e3112e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:42.052492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c5206919-2f96-404f-b289-d78013e3112e None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:42.053697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c5206919-2f96-404f-b289-d78013e3112e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:42.117559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 697/1403] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:42.162571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545398b5-95c4-4479-b558-48fce1dc2235 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:42.163047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545398b5-95c4-4479-b558-48fce1dc2235 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:42.163138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545398b5-95c4-4479-b558-48fce1dc2235 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:42.164562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 707/1404] 10.222.0.33 () {54 vars in 942 bytes} [Mon Jun 20 10:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:42.171154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49d316-99f0-434b-ad70-1897985698b0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:42.171517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49d316-99f0-434b-ad70-1897985698b0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:42.171863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49d316-99f0-434b-ad70-1897985698b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:42.221717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1f49d316-99f0-434b-ad70-1897985698b0 None None] MFA Rules not processed for user `3c133005ca8d4478be8ff4acad5e88cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:42.222984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1f49d316-99f0-434b-ad70-1897985698b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:42.297001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 698/1405] 10.222.0.33 () {58 vars in 1039 bytes} [Mon Jun 20 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:42.306348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545398b5-95c4-4479-b558-48fce1dc2235 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:42.320849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545398b5-95c4-4479-b558-48fce1dc2235 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:42.326978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261ade0-0d51-493b-af8d-f323d5b96ccf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:42.327624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261ade0-0d51-493b-af8d-f323d5b96ccf service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:42.327910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261ade0-0d51-493b-af8d-f323d5b96ccf service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:42.328185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261ade0-0d51-493b-af8d-f323d5b96ccf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:42.336819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261ade0-0d51-493b-af8d-f323d5b96ccf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:42.338450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261ade0-0d51-493b-af8d-f323d5b96ccf service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:42.346944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 708/1406] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 10:57:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 10:57:42.772047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98809e01-941d-42c8-a097-075a4704d23e None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:42.772262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98809e01-941d-42c8-a097-075a4704d23e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:42.772526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98809e01-941d-42c8-a097-075a4704d23e None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:42.774175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 699/1407] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:57:42.779301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a128ea-7595-43de-b241-1722fc6ed28a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:42.779594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a128ea-7595-43de-b241-1722fc6ed28a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:42.779877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a128ea-7595-43de-b241-1722fc6ed28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:42.786421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=67 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:57:42.799989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f0a128ea-7595-43de-b241-1722fc6ed28a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:57:42.801325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f0a128ea-7595-43de-b241-1722fc6ed28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:57:42.810361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 709/1408] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:57:42.855462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98809e01-941d-42c8-a097-075a4704d23e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:57:42.869718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98809e01-941d-42c8-a097-075a4704d23e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:57:42.875319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5c59a-2ff6-42a1-9912-5c633ebd8545 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:57:42.876005 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5c59a-2ff6-42a1-9912-5c633ebd8545 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:57:42.876287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5c59a-2ff6-42a1-9912-5c633ebd8545 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:57:42.876562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5c59a-2ff6-42a1-9912-5c633ebd8545 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:57:42.886082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5c59a-2ff6-42a1-9912-5c633ebd8545 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:57:42.887727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5c59a-2ff6-42a1-9912-5c633ebd8545 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:57:42.893457 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:57:42.898748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 700/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:57:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:57:52.794326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:58:04.544161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b9c68-235f-4110-a947-0b7872c1a32f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:04.544861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b9c68-235f-4110-a947-0b7872c1a32f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:04.544861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b9c68-235f-4110-a947-0b7872c1a32f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:04.546123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 710/1410] 10.222.0.33 () {54 vars in 918 bytes} [Mon Jun 20 10:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:04.551595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f20721-3343-49af-80a2-02a4078e114c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:04.552018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f20721-3343-49af-80a2-02a4078e114c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:04.552146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f20721-3343-49af-80a2-02a4078e114c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:04.571921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-78f20721-3343-49af-80a2-02a4078e114c None None] MFA Rules not processed for user `bff19e37fa2243fda914f7f4e662c8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:04.573105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-78f20721-3343-49af-80a2-02a4078e114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:04.583431 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 701/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jun 20 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:04.597880 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4116767-aded-4b7a-b903-9053f2148ce4 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:04.598198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4116767-aded-4b7a-b903-9053f2148ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:04.598475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4116767-aded-4b7a-b903-9053f2148ce4 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:04.599861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 711/1412] 10.222.0.33 () {54 vars in 918 bytes} [Mon Jun 20 10:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:04.605257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b04d-e852-4cbb-8a9e-d9619d557be6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:04.605575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b04d-e852-4cbb-8a9e-d9619d557be6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:04.605850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b04d-e852-4cbb-8a9e-d9619d557be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:04.625109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9970b04d-e852-4cbb-8a9e-d9619d557be6 None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:04.626345 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9970b04d-e852-4cbb-8a9e-d9619d557be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:04.636570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 702/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jun 20 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:07.138928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6327a7-3964-4324-b905-37de03aaeed8 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:07.139585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6327a7-3964-4324-b905-37de03aaeed8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:07.139585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6327a7-3964-4324-b905-37de03aaeed8 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:07.140876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:07.147284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4121a-5c6c-4aab-96b5-464dce3d359e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:07.148007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4121a-5c6c-4aab-96b5-464dce3d359e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:07.148963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4121a-5c6c-4aab-96b5-464dce3d359e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:07.173682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-44b4121a-5c6c-4aab-96b5-464dce3d359e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:07.174871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-44b4121a-5c6c-4aab-96b5-464dce3d359e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:07.184371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:07.231390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6327a7-3964-4324-b905-37de03aaeed8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:07.247530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6327a7-3964-4324-b905-37de03aaeed8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:07.252987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce459084-1601-43de-bbee-ecf4e968d229 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:07.253646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce459084-1601-43de-bbee-ecf4e968d229 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:07.253911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce459084-1601-43de-bbee-ecf4e968d229 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:07.254214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce459084-1601-43de-bbee-ecf4e968d229 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:07.263086 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce459084-1601-43de-bbee-ecf4e968d229 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:07.264666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce459084-1601-43de-bbee-ecf4e968d229 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:07.275336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 713/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:08.185684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5373b2-6eff-48aa-8454-5d7fe94608a3 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:08.186290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5373b2-6eff-48aa-8454-5d7fe94608a3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:08.186290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5373b2-6eff-48aa-8454-5d7fe94608a3 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:08.187627 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 10:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:08.192888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0119fd-3107-4337-8b94-8bad6b21fb9b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:08.193341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0119fd-3107-4337-8b94-8bad6b21fb9b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:08.193446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0119fd-3107-4337-8b94-8bad6b21fb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:08.213040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6d0119fd-3107-4337-8b94-8bad6b21fb9b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:08.214271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6d0119fd-3107-4337-8b94-8bad6b21fb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:08.223782 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 10:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:08.414355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5373b2-6eff-48aa-8454-5d7fe94608a3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:08.437786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5373b2-6eff-48aa-8454-5d7fe94608a3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:08.446653 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ec315-155b-4722-a4c6-4a8088e839f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:08.447289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ec315-155b-4722-a4c6-4a8088e839f1 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:08.447561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ec315-155b-4722-a4c6-4a8088e839f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:08.447825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ec315-155b-4722-a4c6-4a8088e839f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:08.462356 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ec315-155b-4722-a4c6-4a8088e839f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:08.464130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ec315-155b-4722-a4c6-4a8088e839f1 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:08.474681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 705/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:09.410490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f565c-bbd2-4870-a11c-e56e39f0d5b9 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:09.411090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f565c-bbd2-4870-a11c-e56e39f0d5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:09.411090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f565c-bbd2-4870-a11c-e56e39f0d5b9 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:09.412475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 10:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:09.418791 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fe095-3c06-4058-9e47-d0b0879d47e2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:09.419246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fe095-3c06-4058-9e47-d0b0879d47e2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:09.419374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fe095-3c06-4058-9e47-d0b0879d47e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:09.448512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ae3fe095-3c06-4058-9e47-d0b0879d47e2 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:09.450122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ae3fe095-3c06-4058-9e47-d0b0879d47e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:09.458645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 10:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:09.616763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f565c-bbd2-4870-a11c-e56e39f0d5b9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:09.639285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f565c-bbd2-4870-a11c-e56e39f0d5b9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:09.645403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8574ee-e79a-4048-b4f7-ea3d3b91234b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:09.646048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8574ee-e79a-4048-b4f7-ea3d3b91234b service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:09.646399 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8574ee-e79a-4048-b4f7-ea3d3b91234b service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:09.646672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8574ee-e79a-4048-b4f7-ea3d3b91234b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:09.655761 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8574ee-e79a-4048-b4f7-ea3d3b91234b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:09.657604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8574ee-e79a-4048-b4f7-ea3d3b91234b service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:09.668659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 716/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:10.551697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8b7a0-214c-4574-a405-b1f04b415686 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:10.552622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8b7a0-214c-4574-a405-b1f04b415686 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:10.552622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8b7a0-214c-4574-a405-b1f04b415686 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:10.554967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 10:58:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:10.560824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5335458-d26f-4b4f-bf90-bc960eb560bf None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:10.561270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5335458-d26f-4b4f-bf90-bc960eb560bf None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:10.561377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5335458-d26f-4b4f-bf90-bc960eb560bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:10.581427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f5335458-d26f-4b4f-bf90-bc960eb560bf None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:10.582732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f5335458-d26f-4b4f-bf90-bc960eb560bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:10.592778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 10:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 10:58:10.744995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8b7a0-214c-4574-a405-b1f04b415686 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:10.763628 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8b7a0-214c-4574-a405-b1f04b415686 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:10.771071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71a451-a874-4e4c-a321-a329efe46ef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:10.771719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71a451-a874-4e4c-a321-a329efe46ef5 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:10.771994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71a451-a874-4e4c-a321-a329efe46ef5 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:10.772269 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71a451-a874-4e4c-a321-a329efe46ef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:10.781242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71a451-a874-4e4c-a321-a329efe46ef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:10.782886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71a451-a874-4e4c-a321-a329efe46ef5 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:10.792760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 708/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:11.658987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6021-4998-41c1-9427-bfb3779eff29 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:11.659890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6021-4998-41c1-9427-bfb3779eff29 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:11.659890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd6021-4998-41c1-9427-bfb3779eff29 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:11.662150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 10:58:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:11.667297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6ff76-c5cc-494e-aefd-cfa5e6ffa6af None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:11.667736 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6ff76-c5cc-494e-aefd-cfa5e6ffa6af None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:11.667867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d6ff76-c5cc-494e-aefd-cfa5e6ffa6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:11.686368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-79d6ff76-c5cc-494e-aefd-cfa5e6ffa6af None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:11.687549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-79d6ff76-c5cc-494e-aefd-cfa5e6ffa6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:11.695955 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 10:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:11.854060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd6021-4998-41c1-9427-bfb3779eff29 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:11.877510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd6021-4998-41c1-9427-bfb3779eff29 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:11.885913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9546c0a-3c83-42ea-8bc2-9c5006d293cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:11.886592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546c0a-3c83-42ea-8bc2-9c5006d293cc service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:11.886873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546c0a-3c83-42ea-8bc2-9c5006d293cc service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:11.887146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546c0a-3c83-42ea-8bc2-9c5006d293cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:11.895847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9546c0a-3c83-42ea-8bc2-9c5006d293cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:11.897449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9546c0a-3c83-42ea-8bc2-9c5006d293cc service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:11.907350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 719/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:12.729101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0afe47-6cf1-47ed-8e7b-38cea0684e21 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:12.729664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0afe47-6cf1-47ed-8e7b-38cea0684e21 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:12.729664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0afe47-6cf1-47ed-8e7b-38cea0684e21 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:12.731763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 10:58:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:12.740100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd8568-c04a-4728-a396-382db443a11f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:12.740497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd8568-c04a-4728-a396-382db443a11f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:12.740691 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd8568-c04a-4728-a396-382db443a11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:12.762298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-febd8568-c04a-4728-a396-382db443a11f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:12.763485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-febd8568-c04a-4728-a396-382db443a11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:12.771930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 10:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:12.927639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0afe47-6cf1-47ed-8e7b-38cea0684e21 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:12.944896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0afe47-6cf1-47ed-8e7b-38cea0684e21 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:12.951759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08892b4d-304b-4371-b99a-754c4cebe894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:12.952402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892b4d-304b-4371-b99a-754c4cebe894 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:12.952676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892b4d-304b-4371-b99a-754c4cebe894 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:12.952946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08892b4d-304b-4371-b99a-754c4cebe894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:12.961661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892b4d-304b-4371-b99a-754c4cebe894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:12.963353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08892b4d-304b-4371-b99a-754c4cebe894 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:12.973121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 711/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:13.774074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4cbc6-015a-4e8c-92c3-1b91f6716ef3 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:13.774695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4cbc6-015a-4e8c-92c3-1b91f6716ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:13.774695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4cbc6-015a-4e8c-92c3-1b91f6716ef3 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:13.776201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 10:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:13.782361 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9081-d918-4f4b-ae79-f9d48c0352de None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:13.782806 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9081-d918-4f4b-ae79-f9d48c0352de None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:13.782914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9081-d918-4f4b-ae79-f9d48c0352de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:13.808027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-de7c9081-d918-4f4b-ae79-f9d48c0352de None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:13.809962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-de7c9081-d918-4f4b-ae79-f9d48c0352de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:13.823841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 10:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:13.979445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4cbc6-015a-4e8c-92c3-1b91f6716ef3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:14.004225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4cbc6-015a-4e8c-92c3-1b91f6716ef3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:14.012651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802673d-834c-44d6-bdb6-b2466d8d8c0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:14.013295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802673d-834c-44d6-bdb6-b2466d8d8c0f service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:14.013574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802673d-834c-44d6-bdb6-b2466d8d8c0f service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:14.013848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802673d-834c-44d6-bdb6-b2466d8d8c0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:14.022544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802673d-834c-44d6-bdb6-b2466d8d8c0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:14.024218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802673d-834c-44d6-bdb6-b2466d8d8c0f service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:14.034116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 722/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:14.858815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edff045-d241-4a9c-8988-05e80d733325 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:14.859706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edff045-d241-4a9c-8988-05e80d733325 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:14.859706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edff045-d241-4a9c-8988-05e80d733325 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:14.862031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 10:58:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:14.867912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6385d-38e2-4f4e-a678-caa6f31b8d33 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:14.868439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6385d-38e2-4f4e-a678-caa6f31b8d33 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:14.869138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6385d-38e2-4f4e-a678-caa6f31b8d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:14.894548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-41c6385d-38e2-4f4e-a678-caa6f31b8d33 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:14.895751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-41c6385d-38e2-4f4e-a678-caa6f31b8d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:14.905500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 10:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:15.061652 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edff045-d241-4a9c-8988-05e80d733325 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:15.081122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edff045-d241-4a9c-8988-05e80d733325 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:15.089538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481f72c-f66e-4f06-bd7a-b096376526e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:15.090243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481f72c-f66e-4f06-bd7a-b096376526e1 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:15.090529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481f72c-f66e-4f06-bd7a-b096376526e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:15.090802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481f72c-f66e-4f06-bd7a-b096376526e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:15.099566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481f72c-f66e-4f06-bd7a-b096376526e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:15.101164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481f72c-f66e-4f06-bd7a-b096376526e1 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:15.111378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 714/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:15.903346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aca7b-3a3e-4e67-8af1-2bd03ee1414a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:15.904100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aca7b-3a3e-4e67-8af1-2bd03ee1414a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:15.904100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aca7b-3a3e-4e67-8af1-2bd03ee1414a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:15.906185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 10:58:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:15.912680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79160a5-bf4c-4108-832a-113ef0847e85 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:15.913098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79160a5-bf4c-4108-832a-113ef0847e85 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:15.913224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79160a5-bf4c-4108-832a-113ef0847e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:15.933380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e79160a5-bf4c-4108-832a-113ef0847e85 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:15.934589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e79160a5-bf4c-4108-832a-113ef0847e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:15.943344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 10:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:16.097737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aca7b-3a3e-4e67-8af1-2bd03ee1414a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:16.116358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aca7b-3a3e-4e67-8af1-2bd03ee1414a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:16.122826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183c598-e87d-4538-829f-b441a9105c15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:16.123472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183c598-e87d-4538-829f-b441a9105c15 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:16.123758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183c598-e87d-4538-829f-b441a9105c15 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:16.124034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183c598-e87d-4538-829f-b441a9105c15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:16.132320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183c598-e87d-4538-829f-b441a9105c15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:16.133952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183c598-e87d-4538-829f-b441a9105c15 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:16.143564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 725/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:18.049577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19c143-c278-4b85-b597-6e89f4e859c5 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:18.050513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19c143-c278-4b85-b597-6e89f4e859c5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:18.050513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19c143-c278-4b85-b597-6e89f4e859c5 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:18.052771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:18.059481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450789d-bc7f-4a92-9ee2-87d5871280ff None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:18.059923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450789d-bc7f-4a92-9ee2-87d5871280ff None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:18.060031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450789d-bc7f-4a92-9ee2-87d5871280ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:18.088818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2450789d-bc7f-4a92-9ee2-87d5871280ff None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:18.089955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2450789d-bc7f-4a92-9ee2-87d5871280ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:18.098790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:18.141818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19c143-c278-4b85-b597-6e89f4e859c5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:18.156889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19c143-c278-4b85-b597-6e89f4e859c5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:18.160557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa3166-21cb-4c7c-bd41-d1b0e1c661bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:18.161224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa3166-21cb-4c7c-bd41-d1b0e1c661bf service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:18.161503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa3166-21cb-4c7c-bd41-d1b0e1c661bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:18.161810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa3166-21cb-4c7c-bd41-d1b0e1c661bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:18.171527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa3166-21cb-4c7c-bd41-d1b0e1c661bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:18.172969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa3166-21cb-4c7c-bd41-d1b0e1c661bf service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:18.183278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 717/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:20.345654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ab334-bbc6-44b7-9401-b6fad7c2a512 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:20.346261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ab334-bbc6-44b7-9401-b6fad7c2a512 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:20.346261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ab334-bbc6-44b7-9401-b6fad7c2a512 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:20.347689 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:20.353319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996adc1-8d19-428b-886a-9ddecddc4175 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:20.353760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996adc1-8d19-428b-886a-9ddecddc4175 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:20.353864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996adc1-8d19-428b-886a-9ddecddc4175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:20.371989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6996adc1-8d19-428b-886a-9ddecddc4175 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:20.373214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6996adc1-8d19-428b-886a-9ddecddc4175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:20.381396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:20.419965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ab334-bbc6-44b7-9401-b6fad7c2a512 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:20.433038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ab334-bbc6-44b7-9401-b6fad7c2a512 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:20.437320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7d676-4e61-4fb2-8564-a003b5ec9888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:20.437963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7d676-4e61-4fb2-8564-a003b5ec9888 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:20.438266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7d676-4e61-4fb2-8564-a003b5ec9888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:20.438542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7d676-4e61-4fb2-8564-a003b5ec9888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:20.449702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7d676-4e61-4fb2-8564-a003b5ec9888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:20.451230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7d676-4e61-4fb2-8564-a003b5ec9888 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:20.460890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 728/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:22.776965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd7270-2b0b-4c05-aca5-26e1cbf2a1ab None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:22.777560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd7270-2b0b-4c05-aca5-26e1cbf2a1ab None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:22.777560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd7270-2b0b-4c05-aca5-26e1cbf2a1ab None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:22.778937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:22.784324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbbcbb-c648-4ea3-bb15-b22f60f048a5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:22.784771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbbcbb-c648-4ea3-bb15-b22f60f048a5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:22.784878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbbcbb-c648-4ea3-bb15-b22f60f048a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:22.803585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e2cbbcbb-c648-4ea3-bb15-b22f60f048a5 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:22.804783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e2cbbcbb-c648-4ea3-bb15-b22f60f048a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:22.813231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:22.851976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd7270-2b0b-4c05-aca5-26e1cbf2a1ab None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:22.865288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd7270-2b0b-4c05-aca5-26e1cbf2a1ab None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:22.869813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b214f4-da8a-4044-90a5-ff6173376401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:22.870489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b214f4-da8a-4044-90a5-ff6173376401 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:22.870771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b214f4-da8a-4044-90a5-ff6173376401 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:22.871042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b214f4-da8a-4044-90a5-ff6173376401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:22.881367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b214f4-da8a-4044-90a5-ff6173376401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:22.883693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b214f4-da8a-4044-90a5-ff6173376401 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:22.893639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 720/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:25.030887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2c8ee-0e6c-484c-b3b5-6c78ad691e28 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:25.031474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2c8ee-0e6c-484c-b3b5-6c78ad691e28 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:25.031474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2c8ee-0e6c-484c-b3b5-6c78ad691e28 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:25.032842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:58:25.038284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee2d52-a10b-4cdb-8ece-6173a346a599 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:25.038750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee2d52-a10b-4cdb-8ece-6173a346a599 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:25.038848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee2d52-a10b-4cdb-8ece-6173a346a599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:25.058658 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bcee2d52-a10b-4cdb-8ece-6173a346a599 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:25.059840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bcee2d52-a10b-4cdb-8ece-6173a346a599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:25.069175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:25.109127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2c8ee-0e6c-484c-b3b5-6c78ad691e28 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:25.121408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2c8ee-0e6c-484c-b3b5-6c78ad691e28 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:25.125792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52964e95-d822-4af9-818e-6593757e7625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:25.126454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52964e95-d822-4af9-818e-6593757e7625 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:25.126730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52964e95-d822-4af9-818e-6593757e7625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:25.127008 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52964e95-d822-4af9-818e-6593757e7625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:25.136703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52964e95-d822-4af9-818e-6593757e7625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:25.138125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52964e95-d822-4af9-818e-6593757e7625 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:25.147489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 731/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 10:58:27.330522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdb4be-622b-49be-812b-73f65f78b194 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:27.331304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdb4be-622b-49be-812b-73f65f78b194 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:27.331304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdb4be-622b-49be-812b-73f65f78b194 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:27.332941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:27.339570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0996f-63b9-4a53-944c-5e6ca5c7d23e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:27.340025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0996f-63b9-4a53-944c-5e6ca5c7d23e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:27.340139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0996f-63b9-4a53-944c-5e6ca5c7d23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:27.359666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2cd0996f-63b9-4a53-944c-5e6ca5c7d23e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:27.360883 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2cd0996f-63b9-4a53-944c-5e6ca5c7d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:27.369732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:27.411410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdb4be-622b-49be-812b-73f65f78b194 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:27.428224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdb4be-622b-49be-812b-73f65f78b194 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:27.434751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729b909-5041-4f16-a9c6-d1623b17890d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:27.435735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b909-5041-4f16-a9c6-d1623b17890d service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:27.436131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b909-5041-4f16-a9c6-d1623b17890d service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:27.436546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b909-5041-4f16-a9c6-d1623b17890d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:27.451621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729b909-5041-4f16-a9c6-d1623b17890d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:27.453104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729b909-5041-4f16-a9c6-d1623b17890d service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:27.463807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 723/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:29.585308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34d3b8-810e-4d7e-8124-ae1b2ed4d7bc None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:29.585908 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34d3b8-810e-4d7e-8124-ae1b2ed4d7bc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:29.585908 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34d3b8-810e-4d7e-8124-ae1b2ed4d7bc None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:29.587322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:29.592862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765cf8f-402e-4f72-bf17-a4ab5f6ebf2d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:29.593309 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765cf8f-402e-4f72-bf17-a4ab5f6ebf2d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:29.593419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765cf8f-402e-4f72-bf17-a4ab5f6ebf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:29.613270 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d765cf8f-402e-4f72-bf17-a4ab5f6ebf2d None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:29.614490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d765cf8f-402e-4f72-bf17-a4ab5f6ebf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:29.623852 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:29.665596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34d3b8-810e-4d7e-8124-ae1b2ed4d7bc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:29.677963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34d3b8-810e-4d7e-8124-ae1b2ed4d7bc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:29.683028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c19b9-1b71-4d14-9d00-409c523ce33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:29.684048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c19b9-1b71-4d14-9d00-409c523ce33e service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:29.684452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c19b9-1b71-4d14-9d00-409c523ce33e service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:29.684860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c19b9-1b71-4d14-9d00-409c523ce33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:29.699921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c19b9-1b71-4d14-9d00-409c523ce33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:29.701370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c19b9-1b71-4d14-9d00-409c523ce33e service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:29.711566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 734/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:31.788232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e11975-8034-4f76-9c65-1389eca84146 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:31.788823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e11975-8034-4f76-9c65-1389eca84146 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:31.788823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e11975-8034-4f76-9c65-1389eca84146 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:31.790236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:31.796779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e79ffe-bd0d-4db2-b9a8-9d66ec747e66 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:31.797547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e79ffe-bd0d-4db2-b9a8-9d66ec747e66 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:31.797722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e79ffe-bd0d-4db2-b9a8-9d66ec747e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:31.826321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a8e79ffe-bd0d-4db2-b9a8-9d66ec747e66 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:31.827567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a8e79ffe-bd0d-4db2-b9a8-9d66ec747e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:31.841185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:31.881434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e11975-8034-4f76-9c65-1389eca84146 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:31.893497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e11975-8034-4f76-9c65-1389eca84146 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:31.897463 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753855f-079d-4313-81eb-f1f1eae7c340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:31.898129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753855f-079d-4313-81eb-f1f1eae7c340 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:31.898419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753855f-079d-4313-81eb-f1f1eae7c340 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:31.898691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753855f-079d-4313-81eb-f1f1eae7c340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:31.909672 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753855f-079d-4313-81eb-f1f1eae7c340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:31.911163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753855f-079d-4313-81eb-f1f1eae7c340 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:31.921451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 726/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:34.032222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a08ec0-509e-4a81-907a-5acc5bf78b2a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:34.033102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a08ec0-509e-4a81-907a-5acc5bf78b2a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:34.033289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a08ec0-509e-4a81-907a-5acc5bf78b2a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:34.035563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:34.041598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994734b-be4e-4116-a530-6be6ea436d40 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:34.042046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994734b-be4e-4116-a530-6be6ea436d40 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:34.042191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994734b-be4e-4116-a530-6be6ea436d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:34.049351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=79 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:58:34.066128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8994734b-be4e-4116-a530-6be6ea436d40 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:34.067911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8994734b-be4e-4116-a530-6be6ea436d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:34.079073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:34.117579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a08ec0-509e-4a81-907a-5acc5bf78b2a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:34.125396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=73 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:58:34.134778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a08ec0-509e-4a81-907a-5acc5bf78b2a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:34.139583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ab04f-a62f-45b8-a56d-a8365cc37640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:34.140254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ab04f-a62f-45b8-a56d-a8365cc37640 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:34.140534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ab04f-a62f-45b8-a56d-a8365cc37640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:34.140890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ab04f-a62f-45b8-a56d-a8365cc37640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:34.151668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ab04f-a62f-45b8-a56d-a8365cc37640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:34.153104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ab04f-a62f-45b8-a56d-a8365cc37640 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:34.163391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 737/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:36.187822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458200af-bf0c-446c-b219-e4020158a457 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:36.188376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458200af-bf0c-446c-b219-e4020158a457 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:36.188376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458200af-bf0c-446c-b219-e4020158a457 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:36.189697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:36.196130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f0fa0-e883-422d-9feb-00f8185a8d11 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:36.197184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f0fa0-e883-422d-9feb-00f8185a8d11 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:36.197184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f0fa0-e883-422d-9feb-00f8185a8d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:36.222529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a92f0fa0-e883-422d-9feb-00f8185a8d11 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:36.223742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a92f0fa0-e883-422d-9feb-00f8185a8d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:36.233039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:36.269856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458200af-bf0c-446c-b219-e4020158a457 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:36.286393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458200af-bf0c-446c-b219-e4020158a457 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:36.291307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afdf28-1d15-4887-8f04-87da3d95eb89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:36.292557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afdf28-1d15-4887-8f04-87da3d95eb89 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:36.292959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afdf28-1d15-4887-8f04-87da3d95eb89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:36.293323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afdf28-1d15-4887-8f04-87da3d95eb89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:36.304804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afdf28-1d15-4887-8f04-87da3d95eb89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:36.306309 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afdf28-1d15-4887-8f04-87da3d95eb89 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:36.315781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 729/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:38.371471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328af4f5-4bd6-49e3-8fc4-d4aee8a5aec2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:38.372049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328af4f5-4bd6-49e3-8fc4-d4aee8a5aec2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:38.372049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328af4f5-4bd6-49e3-8fc4-d4aee8a5aec2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:38.373414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 10:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:38.379711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f690eb-b6c6-4b0d-a131-c3ec50e3385b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:38.380168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f690eb-b6c6-4b0d-a131-c3ec50e3385b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:38.380359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f690eb-b6c6-4b0d-a131-c3ec50e3385b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:38.404933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a8f690eb-b6c6-4b0d-a131-c3ec50e3385b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:38.406145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a8f690eb-b6c6-4b0d-a131-c3ec50e3385b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:38.414642 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 10:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:38.451682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328af4f5-4bd6-49e3-8fc4-d4aee8a5aec2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:38.468291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328af4f5-4bd6-49e3-8fc4-d4aee8a5aec2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:38.472450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac5997-aedc-4fca-86af-70ab46a74692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:38.473088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac5997-aedc-4fca-86af-70ab46a74692 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:38.473366 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac5997-aedc-4fca-86af-70ab46a74692 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:38.473636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac5997-aedc-4fca-86af-70ab46a74692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:38.483631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac5997-aedc-4fca-86af-70ab46a74692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:38.485079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ac5997-aedc-4fca-86af-70ab46a74692 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:38.494830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 740/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:58:48.399454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=11 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:58:48.490470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=14 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:58:58.322281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a6777-1893-4fd6-87c5-3c6a460cddb1 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:58.322895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a6777-1893-4fd6-87c5-3c6a460cddb1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:58.322895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a6777-1893-4fd6-87c5-3c6a460cddb1 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:58.324604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 731/1471] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:58.330598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374b294-1731-4caa-8941-3325ac953a1a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:58.331015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374b294-1731-4caa-8941-3325ac953a1a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:58.331153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374b294-1731-4caa-8941-3325ac953a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:58.351041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d374b294-1731-4caa-8941-3325ac953a1a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:58.352259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d374b294-1731-4caa-8941-3325ac953a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:58.361495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 741/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:58.411856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c095f-f9af-49cc-bdc4-d1ec3d10a661 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:58.412144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c095f-f9af-49cc-bdc4-d1ec3d10a661 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:58.412428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c095f-f9af-49cc-bdc4-d1ec3d10a661 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:58.413798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 732/1473] 10.222.0.33 () {54 vars in 962 bytes} [Mon Jun 20 10:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:58:58.419944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba4da-6666-4a71-96be-5e684b41edad None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:58.420278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba4da-6666-4a71-96be-5e684b41edad None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:58.420552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba4da-6666-4a71-96be-5e684b41edad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:58.442010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7d7ba4da-6666-4a71-96be-5e684b41edad None None] MFA Rules not processed for user `e4bf4d9571d541bcbba87094198dc2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:58:58.443249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7d7ba4da-6666-4a71-96be-5e684b41edad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:58:58.451921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 742/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Jun 20 10:58:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:58:58.459435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c095f-f9af-49cc-bdc4-d1ec3d10a661 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:58:58.471869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c095f-f9af-49cc-bdc4-d1ec3d10a661 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:58:58.476294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d48006-0a60-4bc2-8c5f-5a397ac96845 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bf4d9571d541bcbba87094198dc2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:58:58.476940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d48006-0a60-4bc2-8c5f-5a397ac96845 service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:58:58.477227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d48006-0a60-4bc2-8c5f-5a397ac96845 service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:58:58.477506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d48006-0a60-4bc2-8c5f-5a397ac96845 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:58:58.487012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d48006-0a60-4bc2-8c5f-5a397ac96845 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:58:58.488464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d48006-0a60-4bc2-8c5f-5a397ac96845 service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:58:58.498430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 733/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:59:00.609849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abffe91-e9a2-44a4-a405-4b175e0c7b18 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:00.610487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abffe91-e9a2-44a4-a405-4b175e0c7b18 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:00.610487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abffe91-e9a2-44a4-a405-4b175e0c7b18 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:00.611855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 743/1476] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 10:59:00.618418 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97de411-b919-4b6e-9de3-29dcfc6bb008 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:00.619719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97de411-b919-4b6e-9de3-29dcfc6bb008 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:00.619719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97de411-b919-4b6e-9de3-29dcfc6bb008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:00.637859 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e97de411-b919-4b6e-9de3-29dcfc6bb008 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:59:00.639111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e97de411-b919-4b6e-9de3-29dcfc6bb008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:59:00.647633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 734/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:59:00.671529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abffe91-e9a2-44a4-a405-4b175e0c7b18 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:59:00.684362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abffe91-e9a2-44a4-a405-4b175e0c7b18 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:59:00.688704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2c310-27fc-4f40-9abb-f58105c442c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:59:00.689357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2c310-27fc-4f40-9abb-f58105c442c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:00.689637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2c310-27fc-4f40-9abb-f58105c442c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:00.689911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2c310-27fc-4f40-9abb-f58105c442c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:00.690696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2c310-27fc-4f40-9abb-f58105c442c2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:59:00.692489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-70f2c310-27fc-4f40-9abb-f58105c442c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 10:59:00.692489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Jun 20 10:59:00.692489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: Jun 20 10:59:00.693679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2c310-27fc-4f40-9abb-f58105c442c2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:59:00.713284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 744/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Jun 20 10:59:00 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 10:59:02.805774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c197f31-9140-43bf-9ccd-aec1458561c9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:02.806367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c197f31-9140-43bf-9ccd-aec1458561c9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:02.806941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c197f31-9140-43bf-9ccd-aec1458561c9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:02.808639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 735/1479] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:59:02.813896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd4b18-a040-4ae6-9d73-20fca7ac2127 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:02.814382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd4b18-a040-4ae6-9d73-20fca7ac2127 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:02.814483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd4b18-a040-4ae6-9d73-20fca7ac2127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:02.834209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9abd4b18-a040-4ae6-9d73-20fca7ac2127 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:59:02.835433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9abd4b18-a040-4ae6-9d73-20fca7ac2127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:59:02.843474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 745/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:59:02.886736 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c197f31-9140-43bf-9ccd-aec1458561c9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:59:02.904068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c197f31-9140-43bf-9ccd-aec1458561c9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:59:02.910919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552c2fc-52e3-43a3-b0cf-32db3d577a9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:59:02.911817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c2fc-52e3-43a3-b0cf-32db3d577a9f service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:02.912291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c2fc-52e3-43a3-b0cf-32db3d577a9f service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:02.912745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c2fc-52e3-43a3-b0cf-32db3d577a9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:02.924863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552c2fc-52e3-43a3-b0cf-32db3d577a9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:59:02.926521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552c2fc-52e3-43a3-b0cf-32db3d577a9f service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:59:02.936264 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 736/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:59:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:59:05.132842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc2961-4b70-42a9-8de1-57a0498577a8 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:05.133442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc2961-4b70-42a9-8de1-57a0498577a8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:05.133442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc2961-4b70-42a9-8de1-57a0498577a8 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:05.134862 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 746/1482] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:59:05.141241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37e116-32a4-4f1c-bdb8-67e17ee474a6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:05.141939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37e116-32a4-4f1c-bdb8-67e17ee474a6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:05.142162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37e116-32a4-4f1c-bdb8-67e17ee474a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:05.164206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7f37e116-32a4-4f1c-bdb8-67e17ee474a6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:59:05.165351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7f37e116-32a4-4f1c-bdb8-67e17ee474a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:59:05.174930 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 737/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:59:05.212222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc2961-4b70-42a9-8de1-57a0498577a8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:59:05.225043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc2961-4b70-42a9-8de1-57a0498577a8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:59:05.229659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc5054-4029-400c-b6f1-d5f583bc7c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:59:05.230342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc5054-4029-400c-b6f1-d5f583bc7c4e service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:05.230624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc5054-4029-400c-b6f1-d5f583bc7c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:05.230899 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc5054-4029-400c-b6f1-d5f583bc7c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:05.240748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc5054-4029-400c-b6f1-d5f583bc7c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:59:05.242228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc5054-4029-400c-b6f1-d5f583bc7c4e service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:59:05.251838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 747/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:59:07.362590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703acb5c-ffc6-4086-94a1-3f7a3fd6948d None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:07.363412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703acb5c-ffc6-4086-94a1-3f7a3fd6948d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:07.363412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703acb5c-ffc6-4086-94a1-3f7a3fd6948d None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:07.365766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 738/1485] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:59:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:59:07.371187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f6cca-5289-4c7c-9980-d4d3f2f80b73 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:07.371603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f6cca-5289-4c7c-9980-d4d3f2f80b73 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:07.371714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f6cca-5289-4c7c-9980-d4d3f2f80b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:07.389910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0b5f6cca-5289-4c7c-9980-d4d3f2f80b73 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:59:07.391166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0b5f6cca-5289-4c7c-9980-d4d3f2f80b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:59:07.399538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 748/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:59:07.440585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703acb5c-ffc6-4086-94a1-3f7a3fd6948d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:59:07.461074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703acb5c-ffc6-4086-94a1-3f7a3fd6948d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:59:07.466179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99379260-2aa8-4de5-9702-ac9d9b33ae92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:59:07.466797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99379260-2aa8-4de5-9702-ac9d9b33ae92 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:07.467055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99379260-2aa8-4de5-9702-ac9d9b33ae92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:07.467343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99379260-2aa8-4de5-9702-ac9d9b33ae92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:07.477161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99379260-2aa8-4de5-9702-ac9d9b33ae92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:59:07.478767 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99379260-2aa8-4de5-9702-ac9d9b33ae92 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:59:07.488802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 739/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:59:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:59:09.804101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d16195-b6f9-42be-8f7e-92a8db4ee3fc None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:09.804698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d16195-b6f9-42be-8f7e-92a8db4ee3fc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:09.804698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d16195-b6f9-42be-8f7e-92a8db4ee3fc None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:09.806051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 749/1488] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:59:09.812796 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef04bd-3614-4e06-81f0-e11d69251e44 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:09.813564 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef04bd-3614-4e06-81f0-e11d69251e44 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:09.813715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef04bd-3614-4e06-81f0-e11d69251e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:09.843089 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dfef04bd-3614-4e06-81f0-e11d69251e44 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:59:09.844278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dfef04bd-3614-4e06-81f0-e11d69251e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:59:09.859106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 740/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:59:09.904199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d16195-b6f9-42be-8f7e-92a8db4ee3fc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:59:09.918631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d16195-b6f9-42be-8f7e-92a8db4ee3fc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:59:09.924892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171df07-7681-42b9-ba6e-623bb45e5483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:59:09.925547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171df07-7681-42b9-ba6e-623bb45e5483 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:09.925830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171df07-7681-42b9-ba6e-623bb45e5483 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:09.926125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171df07-7681-42b9-ba6e-623bb45e5483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:09.939628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171df07-7681-42b9-ba6e-623bb45e5483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:59:09.941068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171df07-7681-42b9-ba6e-623bb45e5483 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:59:09.951883 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 750/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 10:59:12.241297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a05860-db2e-4065-98f1-4c912a57d370 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:12.241831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a05860-db2e-4065-98f1-4c912a57d370 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:12.241831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a05860-db2e-4065-98f1-4c912a57d370 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:12.243155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 741/1491] 10.222.0.33 () {54 vars in 926 bytes} [Mon Jun 20 10:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 10:59:12.250007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf7e10-5101-468d-9ff9-6ec0d0d94104 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:12.250470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf7e10-5101-468d-9ff9-6ec0d0d94104 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:12.250566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf7e10-5101-468d-9ff9-6ec0d0d94104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:12.273140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b8bf7e10-5101-468d-9ff9-6ec0d0d94104 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 10:59:12.274359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b8bf7e10-5101-468d-9ff9-6ec0d0d94104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 10:59:12.283218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 751/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Mon Jun 20 10:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 10:59:12.323633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a05860-db2e-4065-98f1-4c912a57d370 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 10:59:12.343682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a05860-db2e-4065-98f1-4c912a57d370 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 10:59:12.350241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0cfc8-abec-4f31-859f-cf78504405d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 10:59:12.351206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0cfc8-abec-4f31-859f-cf78504405d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 10:59:12.351584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0cfc8-abec-4f31-859f-cf78504405d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 10:59:12.351962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0cfc8-abec-4f31-859f-cf78504405d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 10:59:12.362748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0cfc8-abec-4f31-859f-cf78504405d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 10:59:12.364497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0cfc8-abec-4f31-859f-cf78504405d2 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 10:59:12.374136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 742/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 10:59:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 10:59:22.267205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=33 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 10:59:22.369848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=33 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:00:37.257783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8ecb4-87d9-4cbe-ae04-4fc02734db03 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.258329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8ecb4-87d9-4cbe-ae04-4fc02734db03 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.258437 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8ecb4-87d9-4cbe-ae04-4fc02734db03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.278816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-fde8ecb4-87d9-4cbe-ae04-4fc02734db03 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:00:37.280015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fde8ecb4-87d9-4cbe-ae04-4fc02734db03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:00:37.288444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 752/1494] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:00:37.298041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea59cc-dd58-45b7-b594-e8302807e0af None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.298592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea59cc-dd58-45b7-b594-e8302807e0af None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.298592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea59cc-dd58-45b7-b594-e8302807e0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.317914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-deea59cc-dd58-45b7-b594-e8302807e0af None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:00:37.319178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-deea59cc-dd58-45b7-b594-e8302807e0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:00:37.328725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 743/1495] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:00:37.333942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8ecb4-87d9-4cbe-ae04-4fc02734db03 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:00:37.345768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8ecb4-87d9-4cbe-ae04-4fc02734db03 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:00:37.350845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6bd15-a1e8-4793-9dfc-262bf314d05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:00:37.351508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6bd15-a1e8-4793-9dfc-262bf314d05b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.351792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6bd15-a1e8-4793-9dfc-262bf314d05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.352066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6bd15-a1e8-4793-9dfc-262bf314d05b admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.352837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6bd15-a1e8-4793-9dfc-262bf314d05b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:00:37.353149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6bd15-a1e8-4793-9dfc-262bf314d05b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:00:37.354205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-05d6bd15-a1e8-4793-9dfc-262bf314d05b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:00:37.354205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Jun 20 11:00:37.354205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: Jun 20 11:00:37.355118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6bd15-a1e8-4793-9dfc-262bf314d05b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:00:37.367407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 753/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:00:37.375430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea59cc-dd58-45b7-b594-e8302807e0af None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:00:37.384823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea59cc-dd58-45b7-b594-e8302807e0af None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:00:37.390202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c960ae6-ec21-4a34-aa4b-b72a8b9f4701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:00:37.390857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c960ae6-ec21-4a34-aa4b-b72a8b9f4701 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.391132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c960ae6-ec21-4a34-aa4b-b72a8b9f4701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.391406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c960ae6-ec21-4a34-aa4b-b72a8b9f4701 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.392096 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c960ae6-ec21-4a34-aa4b-b72a8b9f4701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:00:37.393866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-0c960ae6-ec21-4a34-aa4b-b72a8b9f4701 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:00:37.393866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Jun 20 11:00:37.393866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: Jun 20 11:00:37.395071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c960ae6-ec21-4a34-aa4b-b72a8b9f4701 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:00:37.438445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 744/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:00:37 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:00:37.445200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6bd15-a1e8-4793-9dfc-262bf314d05b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:00:37.456277 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6bd15-a1e8-4793-9dfc-262bf314d05b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:00:37.519049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213d35d-e8fb-41f5-acf9-2f8790336667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:00:37.519939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213d35d-e8fb-41f5-acf9-2f8790336667 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.520351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213d35d-e8fb-41f5-acf9-2f8790336667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.520779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213d35d-e8fb-41f5-acf9-2f8790336667 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.521803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213d35d-e8fb-41f5-acf9-2f8790336667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:00:37.523500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-a213d35d-e8fb-41f5-acf9-2f8790336667 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:00:37.523500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Jun 20 11:00:37.523500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: Jun 20 11:00:37.525284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213d35d-e8fb-41f5-acf9-2f8790336667 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:00:37.552847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 754/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:00:37 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:00:37.558120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c960ae6-ec21-4a34-aa4b-b72a8b9f4701 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:00:37.573913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c960ae6-ec21-4a34-aa4b-b72a8b9f4701 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:00:37.579621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b4fb6-1ace-4233-bcdc-fc012a7105ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:00:37.580277 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b4fb6-1ace-4233-bcdc-fc012a7105ea admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.580555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b4fb6-1ace-4233-bcdc-fc012a7105ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.580828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b4fb6-1ace-4233-bcdc-fc012a7105ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.581559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b4fb6-1ace-4233-bcdc-fc012a7105ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:00:37.582621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-6a6b4fb6-1ace-4233-bcdc-fc012a7105ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:00:37.582621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Jun 20 11:00:37.582621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: Jun 20 11:00:37.583503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b4fb6-1ace-4233-bcdc-fc012a7105ea admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:00:37.597077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 745/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:00:37.601354 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213d35d-e8fb-41f5-acf9-2f8790336667 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:00:37.611947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213d35d-e8fb-41f5-acf9-2f8790336667 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:00:37.617608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b77008-f50c-4267-a57b-8122ad311d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:00:37.618468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b77008-f50c-4267-a57b-8122ad311d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.618765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b77008-f50c-4267-a57b-8122ad311d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.618974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b77008-f50c-4267-a57b-8122ad311d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.619727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b77008-f50c-4267-a57b-8122ad311d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:00:37.620771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-d3b77008-f50c-4267-a57b-8122ad311d40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:00:37.620771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Jun 20 11:00:37.620771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: Jun 20 11:00:37.621647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b77008-f50c-4267-a57b-8122ad311d40 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:00:37.635643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 755/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:00:37.639946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b4fb6-1ace-4233-bcdc-fc012a7105ea admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:00:37.650266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b4fb6-1ace-4233-bcdc-fc012a7105ea admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:00:37.656506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63ab71-443a-4f1e-a305-d870b0136f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:00:37.657396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63ab71-443a-4f1e-a305-d870b0136f03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.657670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63ab71-443a-4f1e-a305-d870b0136f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.657945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63ab71-443a-4f1e-a305-d870b0136f03 admin admin] PATH_INFO: `/v3/projects/89414629b0b9495cbec59f618b386c9f/users/0742dbd0d13f42e99b02b721960d3547/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.674890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63ab71-443a-4f1e-a305-d870b0136f03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89414629b0b9495cbec59f618b386c9f, user_id=0742dbd0d13f42e99b02b721960d3547, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:00:37.675936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-3a63ab71-443a-4f1e-a305-d870b0136f03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:00:37.675936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Jun 20 11:00:37.675936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: Jun 20 11:00:37.677974 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63ab71-443a-4f1e-a305-d870b0136f03 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:00:37.693032 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 746/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:00:37 2022] PUT /identity/v3/projects/89414629b0b9495cbec59f618b386c9f/users/0742dbd0d13f42e99b02b721960d3547/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:00:37.705253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce26bc-81f4-4853-bd8e-3578250f0711 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.705545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce26bc-81f4-4853-bd8e-3578250f0711 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.705820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce26bc-81f4-4853-bd8e-3578250f0711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.734125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bfce26bc-81f4-4853-bd8e-3578250f0711 None None] MFA Rules not processed for user `0742dbd0d13f42e99b02b721960d3547`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:00:37.735378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bfce26bc-81f4-4853-bd8e-3578250f0711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:00:37.806576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 756/1502] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:00:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:00:37.820449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f4b3c-0c32-414e-b4e5-22f426b85eb9 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.820731 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f4b3c-0c32-414e-b4e5-22f426b85eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.821007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f4b3c-0c32-414e-b4e5-22f426b85eb9 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.822470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 747/1503] 10.222.0.33 () {54 vars in 963 bytes} [Mon Jun 20 11:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:00:37.828442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4ac6-d07d-4dda-b88c-32b14bc82a68 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.828740 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4ac6-d07d-4dda-b88c-32b14bc82a68 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.829026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4ac6-d07d-4dda-b88c-32b14bc82a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.847570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-862d4ac6-d07d-4dda-b88c-32b14bc82a68 None None] MFA Rules not processed for user `841e89a6caef4a089edc0f3b1df2fe9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:00:37.848829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-862d4ac6-d07d-4dda-b88c-32b14bc82a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:00:37.895618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:00:37.905054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 757/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Mon Jun 20 11:00:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:00:37.913003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f4b3c-0c32-414e-b4e5-22f426b85eb9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:00:37.924744 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f4b3c-0c32-414e-b4e5-22f426b85eb9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:00:37.928927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdef614-e074-4873-a3e4-94ecff67d53a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:00:37.929569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdef614-e074-4873-a3e4-94ecff67d53a service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.929842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdef614-e074-4873-a3e4-94ecff67d53a service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.930191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdef614-e074-4873-a3e4-94ecff67d53a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.941680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdef614-e074-4873-a3e4-94ecff67d53a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:00:37.943186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdef614-e074-4873-a3e4-94ecff67d53a service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:00:37.953051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 748/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:00:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:00:37.995947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57c3eb-0e82-4dc3-bbeb-467d5e311870 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:37.996236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57c3eb-0e82-4dc3-bbeb-467d5e311870 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:37.996518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57c3eb-0e82-4dc3-bbeb-467d5e311870 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:37.997861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 758/1506] 10.222.0.33 () {52 vars in 829 bytes} [Mon Jun 20 11:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:00:38.001721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdef614-e074-4873-a3e4-94ecff67d53a service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:00:38.068055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdef614-e074-4873-a3e4-94ecff67d53a service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:00:38.073915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:00:38.074830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:38.075111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:38.075368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] PATH_INFO: `/v3/users/0742dbd0d13f42e99b02b721960d3547` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:38.076589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0742dbd0d13f42e99b02b721960d3547)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:00:38.077664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:00:38.077664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Jun 20 11:00:38.077664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: Jun 20 11:00:38.078909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:00:38.149724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0742dbd0d13f42e99b02b721960d3547'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:00:38.150500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0742dbd0d13f42e99b02b721960d3547'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:00:38.162240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0742dbd0d13f42e99b02b721960d3547'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:00:38.163028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0742dbd0d13f42e99b02b721960d3547'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:00:38.176086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0742dbd0d13f42e99b02b721960d3547'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:00:38.193682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:00:38.203809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 749/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:00:38 2022] DELETE /identity/v3/users/0742dbd0d13f42e99b02b721960d3547 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:00:38.208577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4542f9-a5a8-46a7-8e06-ecfa258ad6a3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:38.208858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4542f9-a5a8-46a7-8e06-ecfa258ad6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:38.209127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4542f9-a5a8-46a7-8e06-ecfa258ad6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:38.227246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3c4542f9-a5a8-46a7-8e06-ecfa258ad6a3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:00:38.228461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3c4542f9-a5a8-46a7-8e06-ecfa258ad6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:00:38.290025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 759/1508] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:00:38.300004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:00:38.301459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:00:38.362782 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0101a8-2f4a-4404-adac-6c9656c17388 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:00:38.367085 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e1e9c-17de-49d9-ab08-f1a47b77a5fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:00:38.367739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e1e9c-17de-49d9-ab08-f1a47b77a5fa service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:38.368022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e1e9c-17de-49d9-ab08-f1a47b77a5fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:38.368296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e1e9c-17de-49d9-ab08-f1a47b77a5fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:38.377844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e1e9c-17de-49d9-ab08-f1a47b77a5fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:00:38.379451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e1e9c-17de-49d9-ab08-f1a47b77a5fa service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:00:38.389492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 750/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:00:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:00:38.742401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4542f9-a5a8-46a7-8e06-ecfa258ad6a3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:00:38.745453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3c4542f9-a5a8-46a7-8e06-ecfa258ad6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:00:38.761958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4542f9-a5a8-46a7-8e06-ecfa258ad6a3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:00:38.767109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:00:38.770419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:00:38.770783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:00:38.771060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] PATH_INFO: `/v3/projects/89414629b0b9495cbec59f618b386c9f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:00:38.772460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89414629b0b9495cbec59f618b386c9f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:00:38.774416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:00:38.774416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Jun 20 11:00:38.774416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: Jun 20 11:00:38.776176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:00:38.846931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89414629b0b9495cbec59f618b386c9f', 'request_id': 'req-891570de-88c2-4854-b6fa-f9afc61000e0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:00:38.857334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89414629b0b9495cbec59f618b386c9f', 'request_id': 'req-891570de-88c2-4854-b6fa-f9afc61000e0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:00:38.863838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89414629b0b9495cbec59f618b386c9f', 'request_id': 'req-891570de-88c2-4854-b6fa-f9afc61000e0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:00:38.885301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] The token cache is being invalidate because project 89414629b0b9495cbec59f618b386c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:00:38.886041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:00:38.886962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-891570de-88c2-4854-b6fa-f9afc61000e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:00:38.888653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 760/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:00:38 2022] DELETE /identity/v3/projects/89414629b0b9495cbec59f618b386c9f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:00:48.384733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:00:48.881574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:02:19.494210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984aa7a0-5890-439a-91be-76254dc60980 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:19.494984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984aa7a0-5890-439a-91be-76254dc60980 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:19.494984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984aa7a0-5890-439a-91be-76254dc60980 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:19.496833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 11:02:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:02:19.503054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637c39c-e9ab-42b8-a4a3-cfc499b55ce3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:19.503811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637c39c-e9ab-42b8-a4a3-cfc499b55ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:19.503918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637c39c-e9ab-42b8-a4a3-cfc499b55ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:19.528207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9637c39c-e9ab-42b8-a4a3-cfc499b55ce3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:02:19.529432 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9637c39c-e9ab-42b8-a4a3-cfc499b55ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:19.599470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 11:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:02:19.752995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984aa7a0-5890-439a-91be-76254dc60980 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:02:19.755967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-984aa7a0-5890-439a-91be-76254dc60980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:19.841693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984aa7a0-5890-439a-91be-76254dc60980 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:02:19.847208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f100af1-dd41-47b8-9f33-967fe0a5645f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:02:19.847868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f100af1-dd41-47b8-9f33-967fe0a5645f service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:19.848150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f100af1-dd41-47b8-9f33-967fe0a5645f service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:19.848433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f100af1-dd41-47b8-9f33-967fe0a5645f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:19.856983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f100af1-dd41-47b8-9f33-967fe0a5645f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:02:19.858634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f100af1-dd41-47b8-9f33-967fe0a5645f service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:02:19.868312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 752/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:02:20.702466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbb761-7369-467b-9489-370e926328a3 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:20.703278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbb761-7369-467b-9489-370e926328a3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:20.703278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbb761-7369-467b-9489-370e926328a3 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:20.705527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 11:02:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:02:20.713923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c04f40-0a9d-4864-9f3f-7a3f19b17a8e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:20.714417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c04f40-0a9d-4864-9f3f-7a3f19b17a8e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:20.714578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c04f40-0a9d-4864-9f3f-7a3f19b17a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:20.742944 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-39c04f40-0a9d-4864-9f3f-7a3f19b17a8e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:02:20.744191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-39c04f40-0a9d-4864-9f3f-7a3f19b17a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:20.753916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 11:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:02:20.919037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbb761-7369-467b-9489-370e926328a3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:02:20.934544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbb761-7369-467b-9489-370e926328a3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:02:20.940625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ad395-6836-42c0-9d6f-485fc03dae75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:02:20.941268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ad395-6836-42c0-9d6f-485fc03dae75 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:20.941549 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ad395-6836-42c0-9d6f-485fc03dae75 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:20.941823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ad395-6836-42c0-9d6f-485fc03dae75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:20.950617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ad395-6836-42c0-9d6f-485fc03dae75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:02:20.952234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ad395-6836-42c0-9d6f-485fc03dae75 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:02:20.961995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 763/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:02:21.200548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf1e21-2046-46d7-9df5-f91e729e3f19 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:21.200812 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf1e21-2046-46d7-9df5-f91e729e3f19 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:21.201109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf1e21-2046-46d7-9df5-f91e729e3f19 None None] PATH_INFO: `/v3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:21.202497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 754/1517] 10.222.0.44 () {54 vars in 925 bytes} [Mon Jun 20 11:02:21 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:02:21.209484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820fc8e-8908-41d2-ae8b-2bb20a380b52 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:21.209772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820fc8e-8908-41d2-ae8b-2bb20a380b52 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:21.210057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820fc8e-8908-41d2-ae8b-2bb20a380b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:21.230341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0820fc8e-8908-41d2-ae8b-2bb20a380b52 None None] MFA Rules not processed for user `bff19e37fa2243fda914f7f4e662c8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:02:21.231486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0820fc8e-8908-41d2-ae8b-2bb20a380b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:21.298656 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 764/1518] 10.222.0.44 () {58 vars in 1015 bytes} [Mon Jun 20 11:02:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:02:25.102520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200391ca-82eb-433f-86a4-6ed6fc13a616 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:25.103056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200391ca-82eb-433f-86a4-6ed6fc13a616 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:25.103056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200391ca-82eb-433f-86a4-6ed6fc13a616 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:25.104392 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 755/1519] 10.222.0.33 () {54 vars in 960 bytes} [Mon Jun 20 11:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:02:25.109592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d14b8-135d-43e0-8991-5deebc01c3f3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:25.110034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d14b8-135d-43e0-8991-5deebc01c3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:25.110168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d14b8-135d-43e0-8991-5deebc01c3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:25.129438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-fb9d14b8-135d-43e0-8991-5deebc01c3f3 None None] MFA Rules not processed for user `bff19e37fa2243fda914f7f4e662c8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:02:25.130674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fb9d14b8-135d-43e0-8991-5deebc01c3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:25.139821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 765/1520] 10.222.0.33 () {58 vars in 1057 bytes} [Mon Jun 20 11:02:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:02:25.147395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200391ca-82eb-433f-86a4-6ed6fc13a616 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:02:25.163057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200391ca-82eb-433f-86a4-6ed6fc13a616 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:02:25.168698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077a730-1cef-4a77-95b2-695e5a09a942 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:02:25.169335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077a730-1cef-4a77-95b2-695e5a09a942 service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:25.169619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077a730-1cef-4a77-95b2-695e5a09a942 service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:25.169895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077a730-1cef-4a77-95b2-695e5a09a942 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:25.179024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077a730-1cef-4a77-95b2-695e5a09a942 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:02:25.180721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077a730-1cef-4a77-95b2-695e5a09a942 service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:02:25.190554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 756/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:02:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:02:32.215291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192841a-14cd-4a7b-8fd6-6f7ab8266b2d None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:32.216148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192841a-14cd-4a7b-8fd6-6f7ab8266b2d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:32.216148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192841a-14cd-4a7b-8fd6-6f7ab8266b2d None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:32.218365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 11:02:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:02:32.223536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73e04e-7167-4080-b8e6-aed0b7f2bdc0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:32.223987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73e04e-7167-4080-b8e6-aed0b7f2bdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:32.224082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73e04e-7167-4080-b8e6-aed0b7f2bdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:32.243288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ff73e04e-7167-4080-b8e6-aed0b7f2bdc0 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:02:32.244505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ff73e04e-7167-4080-b8e6-aed0b7f2bdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:32.252953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 11:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:02:32.411558 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192841a-14cd-4a7b-8fd6-6f7ab8266b2d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:02:32.427917 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192841a-14cd-4a7b-8fd6-6f7ab8266b2d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:02:32.434715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690e265-c802-49a6-aae8-4d294068f278 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:02:32.435366 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690e265-c802-49a6-aae8-4d294068f278 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:32.435646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690e265-c802-49a6-aae8-4d294068f278 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:32.435915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690e265-c802-49a6-aae8-4d294068f278 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:32.444849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690e265-c802-49a6-aae8-4d294068f278 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:02:32.447048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690e265-c802-49a6-aae8-4d294068f278 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:02:32.457267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 767/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:02:33.276619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0055cf-c513-4afb-b25b-115a151b3ac4 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:33.277211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0055cf-c513-4afb-b25b-115a151b3ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:33.277211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0055cf-c513-4afb-b25b-115a151b3ac4 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:33.278643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 11:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:02:33.284829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdd42e-974e-4c45-81ff-a8a828922ce2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:33.285566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdd42e-974e-4c45-81ff-a8a828922ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:33.285689 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdd42e-974e-4c45-81ff-a8a828922ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:33.306249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-35cdd42e-974e-4c45-81ff-a8a828922ce2 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:02:33.307489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-35cdd42e-974e-4c45-81ff-a8a828922ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:33.317987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 11:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:02:33.472995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0055cf-c513-4afb-b25b-115a151b3ac4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:02:33.496701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0055cf-c513-4afb-b25b-115a151b3ac4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:02:33.502953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c1ca6-8ab1-4a61-a835-1f0744d98b7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:02:33.503606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c1ca6-8ab1-4a61-a835-1f0744d98b7d service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:33.503894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c1ca6-8ab1-4a61-a835-1f0744d98b7d service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:33.504179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c1ca6-8ab1-4a61-a835-1f0744d98b7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:33.514005 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c1ca6-8ab1-4a61-a835-1f0744d98b7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:02:33.515699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c1ca6-8ab1-4a61-a835-1f0744d98b7d service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:02:33.527023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 759/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:02:37.602494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b96d7-6364-44d4-ab66-24f1ed62bffb None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:37.603086 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b96d7-6364-44d4-ab66-24f1ed62bffb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:37.603086 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b96d7-6364-44d4-ab66-24f1ed62bffb None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:37.604410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 11:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:02:37.611284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15e683-6356-4bbb-a2f9-529e0680da92 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:37.611987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15e683-6356-4bbb-a2f9-529e0680da92 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:37.612183 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15e683-6356-4bbb-a2f9-529e0680da92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:37.639795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5c15e683-6356-4bbb-a2f9-529e0680da92 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:02:37.641011 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5c15e683-6356-4bbb-a2f9-529e0680da92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:37.650229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 11:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:02:37.689465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b96d7-6364-44d4-ab66-24f1ed62bffb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:02:37.691555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f99b96d7-6364-44d4-ab66-24f1ed62bffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:37.751070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b96d7-6364-44d4-ab66-24f1ed62bffb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:02:37.755270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df897b3-4b44-4848-921f-a8b1fc6f931b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:02:37.755919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df897b3-4b44-4848-921f-a8b1fc6f931b service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:37.756227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df897b3-4b44-4848-921f-a8b1fc6f931b service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:37.756479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df897b3-4b44-4848-921f-a8b1fc6f931b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:37.766194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df897b3-4b44-4848-921f-a8b1fc6f931b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:02:37.767680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df897b3-4b44-4848-921f-a8b1fc6f931b service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:02:37.772602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:02:37.777561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 770/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:02:39.812825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de055c6-718c-4918-adf5-55a85c0aa51b None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:39.813645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de055c6-718c-4918-adf5-55a85c0aa51b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:39.813834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de055c6-718c-4918-adf5-55a85c0aa51b None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:39.816049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 11:02:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:02:39.823950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53d2b-b10b-46f8-b46d-279786ebef58 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:39.824399 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53d2b-b10b-46f8-b46d-279786ebef58 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:39.824502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53d2b-b10b-46f8-b46d-279786ebef58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:39.844409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0ab53d2b-b10b-46f8-b46d-279786ebef58 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:02:39.845624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0ab53d2b-b10b-46f8-b46d-279786ebef58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:39.856037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:02:39.901737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de055c6-718c-4918-adf5-55a85c0aa51b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:02:39.914477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de055c6-718c-4918-adf5-55a85c0aa51b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:02:39.918920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc979e2e-9dac-40f9-ab53-2b3522d07ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:02:39.919577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc979e2e-9dac-40f9-ab53-2b3522d07ccf service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:39.919861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc979e2e-9dac-40f9-ab53-2b3522d07ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:39.920132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc979e2e-9dac-40f9-ab53-2b3522d07ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:39.930248 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc979e2e-9dac-40f9-ab53-2b3522d07ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:02:39.931706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc979e2e-9dac-40f9-ab53-2b3522d07ccf service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:02:39.941294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 762/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:02:43.401930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc93a3f-0858-4d17-8bf9-3a5cb3b5714c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:43.402796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc93a3f-0858-4d17-8bf9-3a5cb3b5714c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:43.402796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc93a3f-0858-4d17-8bf9-3a5cb3b5714c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:43.405182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 11:02:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:02:43.411513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217fa35-4e67-4216-905b-0c02f63bdc4c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:43.411983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217fa35-4e67-4216-905b-0c02f63bdc4c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:43.412151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217fa35-4e67-4216-905b-0c02f63bdc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:43.438200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a217fa35-4e67-4216-905b-0c02f63bdc4c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:02:43.439400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a217fa35-4e67-4216-905b-0c02f63bdc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:43.449004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 11:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:02:43.754497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc93a3f-0858-4d17-8bf9-3a5cb3b5714c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:02:43.756533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3fc93a3f-0858-4d17-8bf9-3a5cb3b5714c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:43.816361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc93a3f-0858-4d17-8bf9-3a5cb3b5714c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:02:43.820659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f57e26-f003-4a27-b6f2-f89aab3bd492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:02:43.821324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f57e26-f003-4a27-b6f2-f89aab3bd492 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:43.821610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f57e26-f003-4a27-b6f2-f89aab3bd492 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:43.821892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f57e26-f003-4a27-b6f2-f89aab3bd492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:43.831378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f57e26-f003-4a27-b6f2-f89aab3bd492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:02:43.832818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f57e26-f003-4a27-b6f2-f89aab3bd492 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:02:43.842419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 773/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:02:45.929281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6d3a-23e0-412c-8081-e01eb549d30b None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:45.929896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6d3a-23e0-412c-8081-e01eb549d30b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:45.930117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b6d3a-23e0-412c-8081-e01eb549d30b None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:45.932136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 11:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:02:45.940519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226db25-1e36-424a-8e29-0f22af800580 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:45.941197 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226db25-1e36-424a-8e29-0f22af800580 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:45.941632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226db25-1e36-424a-8e29-0f22af800580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:45.968903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4226db25-1e36-424a-8e29-0f22af800580 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:02:45.970500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4226db25-1e36-424a-8e29-0f22af800580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:45.981356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 11:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:02:46.283654 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b6d3a-23e0-412c-8081-e01eb549d30b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:02:46.287008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b67b6d3a-23e0-412c-8081-e01eb549d30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:02:46.308690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b6d3a-23e0-412c-8081-e01eb549d30b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:02:46.312873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d8d9f-b4ec-49b4-b0d2-4423ed622bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:02:46.313527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d8d9f-b4ec-49b4-b0d2-4423ed622bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:02:46.313813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d8d9f-b4ec-49b4-b0d2-4423ed622bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:02:46.314112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d8d9f-b4ec-49b4-b0d2-4423ed622bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:02:46.324371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d8d9f-b4ec-49b4-b0d2-4423ed622bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:02:46.325824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d8d9f-b4ec-49b4-b0d2-4423ed622bb8 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:02:46.335707 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 765/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:02:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:02:55.960550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=32 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:02:56.331353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=70 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:13.594063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd080d-3dd3-4e58-b2c2-eeddb422d0d0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:13.595085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd080d-3dd3-4e58-b2c2-eeddb422d0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:13.595085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd080d-3dd3-4e58-b2c2-eeddb422d0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:13.620754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c7cd080d-3dd3-4e58-b2c2-eeddb422d0d0 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:13.621943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c7cd080d-3dd3-4e58-b2c2-eeddb422d0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:13.631621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 775/1540] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:13.641411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b701289-7980-41b3-a66d-c63a6d0581b9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:13.642549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b701289-7980-41b3-a66d-c63a6d0581b9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:13.643137 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b701289-7980-41b3-a66d-c63a6d0581b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:13.672979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9b701289-7980-41b3-a66d-c63a6d0581b9 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:13.674263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9b701289-7980-41b3-a66d-c63a6d0581b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:13.686674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 766/1541] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:13.694570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd080d-3dd3-4e58-b2c2-eeddb422d0d0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:13.712673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd080d-3dd3-4e58-b2c2-eeddb422d0d0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:13.723331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f753d88-5d74-4442-b10d-882f7349ab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:13.724050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f753d88-5d74-4442-b10d-882f7349ab1f admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:13.724332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f753d88-5d74-4442-b10d-882f7349ab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:13.724608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f753d88-5d74-4442-b10d-882f7349ab1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:13.725454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f753d88-5d74-4442-b10d-882f7349ab1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:13.725771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f753d88-5d74-4442-b10d-882f7349ab1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:13.727501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f753d88-5d74-4442-b10d-882f7349ab1f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:13.749362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 776/1542] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:13.757718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b701289-7980-41b3-a66d-c63a6d0581b9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:13.773168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b701289-7980-41b3-a66d-c63a6d0581b9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:13.778292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f97e5-d6f9-4e00-a083-6b73bce0f85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:13.778987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f97e5-d6f9-4e00-a083-6b73bce0f85c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:13.779278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f97e5-d6f9-4e00-a083-6b73bce0f85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:13.779685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f97e5-d6f9-4e00-a083-6b73bce0f85c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:13.780465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f97e5-d6f9-4e00-a083-6b73bce0f85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:13.782749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2f97e5-d6f9-4e00-a083-6b73bce0f85c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:13.833357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 767/1543] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:13 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:13.837615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f753d88-5d74-4442-b10d-882f7349ab1f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:13.852354 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f753d88-5d74-4442-b10d-882f7349ab1f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:13.869782 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af036a-f956-40e5-903a-52c0d35b7952 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:13.870205 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af036a-f956-40e5-903a-52c0d35b7952 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:13.870515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af036a-f956-40e5-903a-52c0d35b7952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:13.903206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-24af036a-f956-40e5-903a-52c0d35b7952 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:13.904345 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-24af036a-f956-40e5-903a-52c0d35b7952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:13.976267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7a4b6-d85e-452a-8247-c3903763a6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:13.977050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7a4b6-d85e-452a-8247-c3903763a6ce admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:13.977338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7a4b6-d85e-452a-8247-c3903763a6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:13.977616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7a4b6-d85e-452a-8247-c3903763a6ce admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:13.982859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7a4b6-d85e-452a-8247-c3903763a6ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:13.985919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7a4b6-d85e-452a-8247-c3903763a6ce admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.027561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 777/1544] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:13 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:14.031493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3336175-729e-435f-93b9-0571ea686f78 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.031832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3336175-729e-435f-93b9-0571ea686f78 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.032080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3336175-729e-435f-93b9-0571ea686f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.046633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 768/1545] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.047986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1b33e-c614-494e-8b95-f35fba87ac99 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.048456 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1b33e-c614-494e-8b95-f35fba87ac99 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.048737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1b33e-c614-494e-8b95-f35fba87ac99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.056198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c3336175-729e-435f-93b9-0571ea686f78 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.057956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c3336175-729e-435f-93b9-0571ea686f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.068287 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 778/1546] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.070136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3b4c9-50f8-4d0b-b63e-1176315d5f0c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.071147 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3b4c9-50f8-4d0b-b63e-1176315d5f0c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.071610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3b4c9-50f8-4d0b-b63e-1176315d5f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.083446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-40d1b33e-c614-494e-8b95-f35fba87ac99 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.084786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-40d1b33e-c614-494e-8b95-f35fba87ac99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.093435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 769/1547] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.094290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1b33e-c614-494e-8b95-f35fba87ac99 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.094795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a2b3b4c9-50f8-4d0b-b63e-1176315d5f0c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.096331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a2b3b4c9-50f8-4d0b-b63e-1176315d5f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.104825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1b33e-c614-494e-8b95-f35fba87ac99 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.106668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 779/1548] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.108389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c9025-a313-49c7-bb26-5f97b946339c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.108885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c9025-a313-49c7-bb26-5f97b946339c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.109232 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c9025-a313-49c7-bb26-5f97b946339c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.110708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aaee71-d146-4fe4-9ef5-969eec367596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.111561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaee71-d146-4fe4-9ef5-969eec367596 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.112014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaee71-d146-4fe4-9ef5-969eec367596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.112467 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaee71-d146-4fe4-9ef5-969eec367596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.113211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aaee71-d146-4fe4-9ef5-969eec367596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.115304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aaee71-d146-4fe4-9ef5-969eec367596 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.130020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 770/1549] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:14.130020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891e7d-7eb5-4106-b453-e5fb69a9340a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.130020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891e7d-7eb5-4106-b453-e5fb69a9340a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.130020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11891e7d-7eb5-4106-b453-e5fb69a9340a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.131675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c65c9025-a313-49c7-bb26-5f97b946339c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.132940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c65c9025-a313-49c7-bb26-5f97b946339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.141853 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 780/1550] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.152535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0d52-ecff-4630-9f9b-b2f09e67b290 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.152758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-11891e7d-7eb5-4106-b453-e5fb69a9340a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.152858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0d52-ecff-4630-9f9b-b2f09e67b290 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.153046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a0d52-ecff-4630-9f9b-b2f09e67b290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.153742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-11891e7d-7eb5-4106-b453-e5fb69a9340a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.162054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 771/1551] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.166867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f7f93-296e-4cf9-a97d-41d1440a621b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.167154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f7f93-296e-4cf9-a97d-41d1440a621b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.167484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f7f93-296e-4cf9-a97d-41d1440a621b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.173294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cf6a0d52-ecff-4630-9f9b-b2f09e67b290 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.174877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cf6a0d52-ecff-4630-9f9b-b2f09e67b290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.185644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-561f7f93-296e-4cf9-a97d-41d1440a621b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.186501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 781/1552] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.187466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-561f7f93-296e-4cf9-a97d-41d1440a621b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.187604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c45a1-73a0-4de6-b649-01de8ad49bc7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.187901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c45a1-73a0-4de6-b649-01de8ad49bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.188181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c45a1-73a0-4de6-b649-01de8ad49bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.196312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 772/1553] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.197616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343a472-816c-4284-8df8-2d0b1bafa546 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.197904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343a472-816c-4284-8df8-2d0b1bafa546 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.198228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343a472-816c-4284-8df8-2d0b1bafa546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.208699 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7d8c45a1-73a0-4de6-b649-01de8ad49bc7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.210620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7d8c45a1-73a0-4de6-b649-01de8ad49bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.219240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 782/1554] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.220084 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c45a1-73a0-4de6-b649-01de8ad49bc7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.224169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3343a472-816c-4284-8df8-2d0b1bafa546 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.225369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3343a472-816c-4284-8df8-2d0b1bafa546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.230235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c45a1-73a0-4de6-b649-01de8ad49bc7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.234296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 773/1555] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.234771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40f697-16f6-4bb8-a78b-ae06e025d7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.235504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40f697-16f6-4bb8-a78b-ae06e025d7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.235791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40f697-16f6-4bb8-a78b-ae06e025d7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.236013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327f094-aa3f-4045-9efd-2c8c773fcc28 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.236110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40f697-16f6-4bb8-a78b-ae06e025d7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.236468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327f094-aa3f-4045-9efd-2c8c773fcc28 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.236779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40f697-16f6-4bb8-a78b-ae06e025d7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.236929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327f094-aa3f-4045-9efd-2c8c773fcc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.238585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40f697-16f6-4bb8-a78b-ae06e025d7b5 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.248955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 783/1556] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:14.249838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40f697-16f6-4bb8-a78b-ae06e025d7b5 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.256036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a327f094-aa3f-4045-9efd-2c8c773fcc28 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.257247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a327f094-aa3f-4045-9efd-2c8c773fcc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.261577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40f697-16f6-4bb8-a78b-ae06e025d7b5 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.265430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 774/1557] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.266178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48ceea-699a-41be-9f2e-f427f5ebfcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.266724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192844-3dc9-49fa-b6e7-1f2ad649bb41 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.266823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48ceea-699a-41be-9f2e-f427f5ebfcd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.267009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192844-3dc9-49fa-b6e7-1f2ad649bb41 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.267107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48ceea-699a-41be-9f2e-f427f5ebfcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.267285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192844-3dc9-49fa-b6e7-1f2ad649bb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.267376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48ceea-699a-41be-9f2e-f427f5ebfcd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.268237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48ceea-699a-41be-9f2e-f427f5ebfcd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:14.268552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48ceea-699a-41be-9f2e-f427f5ebfcd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.270919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48ceea-699a-41be-9f2e-f427f5ebfcd9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.285657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e8192844-3dc9-49fa-b6e7-1f2ad649bb41 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.286145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 784/1558] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:14.287025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48ceea-699a-41be-9f2e-f427f5ebfcd9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.287161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e8192844-3dc9-49fa-b6e7-1f2ad649bb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.295929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 775/1559] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.297331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc34ccf-7558-4a32-9e3c-52b92e16c7de None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.297597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc34ccf-7558-4a32-9e3c-52b92e16c7de None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.297903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc34ccf-7558-4a32-9e3c-52b92e16c7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.299047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48ceea-699a-41be-9f2e-f427f5ebfcd9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.305747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe6eec-578f-495c-8ec1-2f4eed0820a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.306454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe6eec-578f-495c-8ec1-2f4eed0820a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.306767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe6eec-578f-495c-8ec1-2f4eed0820a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.307253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe6eec-578f-495c-8ec1-2f4eed0820a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.307781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe6eec-578f-495c-8ec1-2f4eed0820a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:14.308294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe6eec-578f-495c-8ec1-2f4eed0820a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.310045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe6eec-578f-495c-8ec1-2f4eed0820a0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.319278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0bc34ccf-7558-4a32-9e3c-52b92e16c7de None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.320475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0bc34ccf-7558-4a32-9e3c-52b92e16c7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.322605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 785/1560] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:14.323356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe6eec-578f-495c-8ec1-2f4eed0820a0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.329203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 776/1561] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.330108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc34ccf-7558-4a32-9e3c-52b92e16c7de None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.337608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe6eec-578f-495c-8ec1-2f4eed0820a0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.340803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc34ccf-7558-4a32-9e3c-52b92e16c7de None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.342617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016601b-985d-41ff-9fde-a466260e774e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.343382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016601b-985d-41ff-9fde-a466260e774e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.343668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016601b-985d-41ff-9fde-a466260e774e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.343954 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016601b-985d-41ff-9fde-a466260e774e admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.344678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016601b-985d-41ff-9fde-a466260e774e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:14.344990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016601b-985d-41ff-9fde-a466260e774e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.346201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25187da1-c1a5-428e-8066-51098efed1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.346632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016601b-985d-41ff-9fde-a466260e774e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.346861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25187da1-c1a5-428e-8066-51098efed1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.347140 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25187da1-c1a5-428e-8066-51098efed1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.347410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25187da1-c1a5-428e-8066-51098efed1fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.348186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25187da1-c1a5-428e-8066-51098efed1fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:14.348495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25187da1-c1a5-428e-8066-51098efed1fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.349698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-25187da1-c1a5-428e-8066-51098efed1fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:04:14.349698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Jun 20 11:04:14.349698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: Jun 20 11:04:14.350620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25187da1-c1a5-428e-8066-51098efed1fe admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.367212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 786/1562] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:14.367212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016601b-985d-41ff-9fde-a466260e774e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.372500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016601b-985d-41ff-9fde-a466260e774e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.377056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997b4bf-3114-4d6f-9e77-f74c3e30ae74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.377931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997b4bf-3114-4d6f-9e77-f74c3e30ae74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.378253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997b4bf-3114-4d6f-9e77-f74c3e30ae74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.378534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997b4bf-3114-4d6f-9e77-f74c3e30ae74 admin admin] PATH_INFO: `/v3/projects/44ee128f927f4dc685c1a9a4bd0d6b55/users/16aecb7081804b929dd581903c2d6166/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.379212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 777/1563] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:14.380399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25187da1-c1a5-428e-8066-51098efed1fe admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.393622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997b4bf-3114-4d6f-9e77-f74c3e30ae74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ee128f927f4dc685c1a9a4bd0d6b55, user_id=16aecb7081804b929dd581903c2d6166, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.394710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-4997b4bf-3114-4d6f-9e77-f74c3e30ae74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:04:14.394710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Jun 20 11:04:14.394710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: Jun 20 11:04:14.396629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997b4bf-3114-4d6f-9e77-f74c3e30ae74 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.397262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25187da1-c1a5-428e-8066-51098efed1fe admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.402150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103f6ee-6429-4a0b-a290-9c957148e287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.402824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f6ee-6429-4a0b-a290-9c957148e287 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.403105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f6ee-6429-4a0b-a290-9c957148e287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.403381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f6ee-6429-4a0b-a290-9c957148e287 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.404170 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103f6ee-6429-4a0b-a290-9c957148e287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:14.404420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103f6ee-6429-4a0b-a290-9c957148e287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.406154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103f6ee-6429-4a0b-a290-9c957148e287 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.410532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 787/1564] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:14 2022] PUT /identity/v3/projects/44ee128f927f4dc685c1a9a4bd0d6b55/users/16aecb7081804b929dd581903c2d6166/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:14.411403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997b4bf-3114-4d6f-9e77-f74c3e30ae74 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.419321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 778/1565] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:14.423527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2420d5e-c719-4142-bff0-96e8f24455fb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.423816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2420d5e-c719-4142-bff0-96e8f24455fb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.424094 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2420d5e-c719-4142-bff0-96e8f24455fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.437732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:14.442600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e2420d5e-c719-4142-bff0-96e8f24455fb None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.443800 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e2420d5e-c719-4142-bff0-96e8f24455fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.476677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997b4bf-3114-4d6f-9e77-f74c3e30ae74 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.481698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99403d-9074-4fc3-8adb-c68219a81d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.482406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99403d-9074-4fc3-8adb-c68219a81d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.482703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99403d-9074-4fc3-8adb-c68219a81d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.483048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99403d-9074-4fc3-8adb-c68219a81d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.483753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99403d-9074-4fc3-8adb-c68219a81d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.484839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-7d99403d-9074-4fc3-8adb-c68219a81d48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:04:14.484839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Jun 20 11:04:14.484839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: Jun 20 11:04:14.486051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d99403d-9074-4fc3-8adb-c68219a81d48 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.490592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:14.508071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 779/1566] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.508911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2420d5e-c719-4142-bff0-96e8f24455fb None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.517242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 788/1567] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:14.517242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99403d-9074-4fc3-8adb-c68219a81d48 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.522006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2420d5e-c719-4142-bff0-96e8f24455fb None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.577803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99403d-9074-4fc3-8adb-c68219a81d48 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.580567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c51b1d-7967-4865-bf1b-9dcbfc6f05d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.581309 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c51b1d-7967-4865-bf1b-9dcbfc6f05d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.581588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c51b1d-7967-4865-bf1b-9dcbfc6f05d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.581861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c51b1d-7967-4865-bf1b-9dcbfc6f05d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.582357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b32a7-1337-426c-80f6-5db6f89411a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.582585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c51b1d-7967-4865-bf1b-9dcbfc6f05d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.583030 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b32a7-1337-426c-80f6-5db6f89411a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.583307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b32a7-1337-426c-80f6-5db6f89411a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.583581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b32a7-1337-426c-80f6-5db6f89411a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.584307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b32a7-1337-426c-80f6-5db6f89411a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:14.584521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c51b1d-7967-4865-bf1b-9dcbfc6f05d9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.584625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b32a7-1337-426c-80f6-5db6f89411a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.586360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b32a7-1337-426c-80f6-5db6f89411a4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.599382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 789/1568] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:14.600342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b32a7-1337-426c-80f6-5db6f89411a4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.612494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b32a7-1337-426c-80f6-5db6f89411a4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.619179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bc5d8-136e-4674-a9de-b820511a13ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.620057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bc5d8-136e-4674-a9de-b820511a13ac admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.620360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bc5d8-136e-4674-a9de-b820511a13ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.620807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 780/1569] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:14.621301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bc5d8-136e-4674-a9de-b820511a13ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.621555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bc5d8-136e-4674-a9de-b820511a13ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.623146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c51b1d-7967-4865-bf1b-9dcbfc6f05d9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.623497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bc5d8-136e-4674-a9de-b820511a13ac admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.650131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 790/1570] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:14.650741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64217310-c38c-4017-9d16-94e48eb40d31 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.651036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64217310-c38c-4017-9d16-94e48eb40d31 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.651315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64217310-c38c-4017-9d16-94e48eb40d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.681470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-64217310-c38c-4017-9d16-94e48eb40d31 None None] MFA Rules not processed for user `16aecb7081804b929dd581903c2d6166`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:14.682635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-64217310-c38c-4017-9d16-94e48eb40d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:14.689894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c51b1d-7967-4865-bf1b-9dcbfc6f05d9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.750991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 791/1571] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:14.751231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5ba55-ab8f-45ee-a5a9-913b4024afac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.751534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ba55-ab8f-45ee-a5a9-913b4024afac admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.751655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64217310-c38c-4017-9d16-94e48eb40d31 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.751826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ba55-ab8f-45ee-a5a9-913b4024afac admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.752103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ba55-ab8f-45ee-a5a9-913b4024afac admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.752944 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5ba55-ab8f-45ee-a5a9-913b4024afac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.754963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5ba55-ab8f-45ee-a5a9-913b4024afac admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.765161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64217310-c38c-4017-9d16-94e48eb40d31 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.770024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78763672-52bc-4786-b9be-ffa57e65a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.770781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78763672-52bc-4786-b9be-ffa57e65a793 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.771131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78763672-52bc-4786-b9be-ffa57e65a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.771524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78763672-52bc-4786-b9be-ffa57e65a793 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.772332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78763672-52bc-4786-b9be-ffa57e65a793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.774593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78763672-52bc-4786-b9be-ffa57e65a793 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.779227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:14.782822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 781/1572] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:14.787078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5ba55-ab8f-45ee-a5a9-913b4024afac admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.796232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:14.804096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 792/1573] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:14.805072 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78763672-52bc-4786-b9be-ffa57e65a793 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.854910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5ba55-ab8f-45ee-a5a9-913b4024afac admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.872079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78763672-52bc-4786-b9be-ffa57e65a793 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.876902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd6b9a-c5e8-4a9d-9fb8-1189aa27ea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.877577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd6b9a-c5e8-4a9d-9fb8-1189aa27ea75 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.877993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd6b9a-c5e8-4a9d-9fb8-1189aa27ea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.878306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd6b9a-c5e8-4a9d-9fb8-1189aa27ea75 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.879207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd6b9a-c5e8-4a9d-9fb8-1189aa27ea75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.880931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd6b9a-c5e8-4a9d-9fb8-1189aa27ea75 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.901794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 793/1574] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:14.906124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd6b9a-c5e8-4a9d-9fb8-1189aa27ea75 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.916284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd6b9a-c5e8-4a9d-9fb8-1189aa27ea75 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.918349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765658-57c6-4958-9926-87f4778cd488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.919031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765658-57c6-4958-9926-87f4778cd488 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.919318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765658-57c6-4958-9926-87f4778cd488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.919601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765658-57c6-4958-9926-87f4778cd488 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.920382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b765658-57c6-4958-9926-87f4778cd488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:14.920697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b765658-57c6-4958-9926-87f4778cd488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.921020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19296479-f020-4167-b4dd-2e529054a380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.921685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19296479-f020-4167-b4dd-2e529054a380 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.921973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19296479-f020-4167-b4dd-2e529054a380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.922290 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19296479-f020-4167-b4dd-2e529054a380 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.922404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b765658-57c6-4958-9926-87f4778cd488 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.922992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19296479-f020-4167-b4dd-2e529054a380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.924900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19296479-f020-4167-b4dd-2e529054a380 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.934426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 782/1575] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:14.935316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765658-57c6-4958-9926-87f4778cd488 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.947171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765658-57c6-4958-9926-87f4778cd488 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:14.955157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 794/1576] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:14.955306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac55db4-6d36-4639-b627-4503727d0e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:14.955306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac55db4-6d36-4639-b627-4503727d0e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:14.955541 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19296479-f020-4167-b4dd-2e529054a380 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.955650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac55db4-6d36-4639-b627-4503727d0e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:14.955650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac55db4-6d36-4639-b627-4503727d0e6d admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:14.955650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac55db4-6d36-4639-b627-4503727d0e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:14.955650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-aac55db4-6d36-4639-b627-4503727d0e6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:04:14.955650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Jun 20 11:04:14.955650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: Jun 20 11:04:14.956594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac55db4-6d36-4639-b627-4503727d0e6d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:14.979034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 783/1577] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:14.979964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac55db4-6d36-4639-b627-4503727d0e6d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:14.990004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac55db4-6d36-4639-b627-4503727d0e6d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.018461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19296479-f020-4167-b4dd-2e529054a380 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.023464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aac44a-6168-4015-8851-c68aa38fcaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.024140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aac44a-6168-4015-8851-c68aa38fcaa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.024421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aac44a-6168-4015-8851-c68aa38fcaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.024696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aac44a-6168-4015-8851-c68aa38fcaa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.025389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aac44a-6168-4015-8851-c68aa38fcaa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.027422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aac44a-6168-4015-8851-c68aa38fcaa5 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.048684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 795/1578] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:15.049570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aac44a-6168-4015-8851-c68aa38fcaa5 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.055817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353832a7-5eb9-4e31-81eb-cca72d4893ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.056501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353832a7-5eb9-4e31-81eb-cca72d4893ba admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.056787 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353832a7-5eb9-4e31-81eb-cca72d4893ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.057068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353832a7-5eb9-4e31-81eb-cca72d4893ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.057919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353832a7-5eb9-4e31-81eb-cca72d4893ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.060000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353832a7-5eb9-4e31-81eb-cca72d4893ba admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.062044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aac44a-6168-4015-8851-c68aa38fcaa5 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.067415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115c6ee-6d60-4685-bd17-21de01dacd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.068079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115c6ee-6d60-4685-bd17-21de01dacd2c admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.068357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115c6ee-6d60-4685-bd17-21de01dacd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.068639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115c6ee-6d60-4685-bd17-21de01dacd2c admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.069313 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115c6ee-6d60-4685-bd17-21de01dacd2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.071924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115c6ee-6d60-4685-bd17-21de01dacd2c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.088564 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 784/1579] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:14 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:15.089602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353832a7-5eb9-4e31-81eb-cca72d4893ba admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.092827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 796/1580] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:15 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:15.093641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115c6ee-6d60-4685-bd17-21de01dacd2c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.103590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115c6ee-6d60-4685-bd17-21de01dacd2c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.107946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:15.128827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:15.157286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353832a7-5eb9-4e31-81eb-cca72d4893ba admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.161501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc78c3c-5c46-48c2-9551-994ef3ac4c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.161866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05bfb0-b75e-4cc6-92ba-bd575d76d557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.162215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc78c3c-5c46-48c2-9551-994ef3ac4c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.162498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc78c3c-5c46-48c2-9551-994ef3ac4c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.162610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05bfb0-b75e-4cc6-92ba-bd575d76d557 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.162777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc78c3c-5c46-48c2-9551-994ef3ac4c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.162884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05bfb0-b75e-4cc6-92ba-bd575d76d557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.163102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05bfb0-b75e-4cc6-92ba-bd575d76d557 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.163491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc78c3c-5c46-48c2-9551-994ef3ac4c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.163797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05bfb0-b75e-4cc6-92ba-bd575d76d557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.165145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc78c3c-5c46-48c2-9551-994ef3ac4c97 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.165757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05bfb0-b75e-4cc6-92ba-bd575d76d557 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.175775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 797/1581] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.176539 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc78c3c-5c46-48c2-9551-994ef3ac4c97 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.186626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc78c3c-5c46-48c2-9551-994ef3ac4c97 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.187893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 785/1582] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:15 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:15.188721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05bfb0-b75e-4cc6-92ba-bd575d76d557 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.194513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae8a37-696f-4ef0-98e6-53c6d87516de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.195168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae8a37-696f-4ef0-98e6-53c6d87516de admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.195464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae8a37-696f-4ef0-98e6-53c6d87516de admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.196103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fae8a37-696f-4ef0-98e6-53c6d87516de admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.196835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae8a37-696f-4ef0-98e6-53c6d87516de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.198918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fae8a37-696f-4ef0-98e6-53c6d87516de admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.201347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05bfb0-b75e-4cc6-92ba-bd575d76d557 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.206367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ee14-2fb5-4078-8b5b-5185b9ae9913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.207022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ee14-2fb5-4078-8b5b-5185b9ae9913 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.207305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ee14-2fb5-4078-8b5b-5185b9ae9913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.207566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ee14-2fb5-4078-8b5b-5185b9ae9913 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.208266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ee14-2fb5-4078-8b5b-5185b9ae9913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.210307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ee14-2fb5-4078-8b5b-5185b9ae9913 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.228949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 798/1583] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:15 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:15.228949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae8a37-696f-4ef0-98e6-53c6d87516de admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.237772 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 786/1584] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:15 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:15.237772 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ee14-2fb5-4078-8b5b-5185b9ae9913 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.246367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ee14-2fb5-4078-8b5b-5185b9ae9913 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.289504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fae8a37-696f-4ef0-98e6-53c6d87516de admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.294445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea584fb-37d8-4f3f-be06-201a1f8ccbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.295126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea584fb-37d8-4f3f-be06-201a1f8ccbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.295409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea584fb-37d8-4f3f-be06-201a1f8ccbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.295686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea584fb-37d8-4f3f-be06-201a1f8ccbd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.296445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea584fb-37d8-4f3f-be06-201a1f8ccbd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.298095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea584fb-37d8-4f3f-be06-201a1f8ccbd6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.301595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8b736-e74d-4e47-8aa2-475da127de55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.302306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8b736-e74d-4e47-8aa2-475da127de55 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.302583 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8b736-e74d-4e47-8aa2-475da127de55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.302865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8b736-e74d-4e47-8aa2-475da127de55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.303599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8b736-e74d-4e47-8aa2-475da127de55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.305336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8b736-e74d-4e47-8aa2-475da127de55 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.308449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 799/1585] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.309240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea584fb-37d8-4f3f-be06-201a1f8ccbd6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.319053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 787/1586] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.319053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8b736-e74d-4e47-8aa2-475da127de55 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.319336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea584fb-37d8-4f3f-be06-201a1f8ccbd6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.324832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e2988-a58d-45fd-94bc-05f7e247326e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.325531 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e2988-a58d-45fd-94bc-05f7e247326e admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.325808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e2988-a58d-45fd-94bc-05f7e247326e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.326081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e2988-a58d-45fd-94bc-05f7e247326e admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.326783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e2988-a58d-45fd-94bc-05f7e247326e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.327531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8b736-e74d-4e47-8aa2-475da127de55 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.328694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e2988-a58d-45fd-94bc-05f7e247326e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.332275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37d6bd-bca9-46b9-94f6-dc92f41368c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.332935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37d6bd-bca9-46b9-94f6-dc92f41368c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.333249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37d6bd-bca9-46b9-94f6-dc92f41368c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.333527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37d6bd-bca9-46b9-94f6-dc92f41368c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.334273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37d6bd-bca9-46b9-94f6-dc92f41368c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.336058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37d6bd-bca9-46b9-94f6-dc92f41368c3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.348559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 788/1587] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.349376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37d6bd-bca9-46b9-94f6-dc92f41368c3 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.351675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 800/1588] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:15 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:15.352423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e2988-a58d-45fd-94bc-05f7e247326e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.361431 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37d6bd-bca9-46b9-94f6-dc92f41368c3 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.362880 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e2988-a58d-45fd-94bc-05f7e247326e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.367027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcec04-fdf8-4309-91ea-9a26fccdcca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.367520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782be15-e170-4777-a9d0-8cd5dbb80e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.367837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcec04-fdf8-4309-91ea-9a26fccdcca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.368121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcec04-fdf8-4309-91ea-9a26fccdcca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.368245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782be15-e170-4777-a9d0-8cd5dbb80e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.368393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcec04-fdf8-4309-91ea-9a26fccdcca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.368493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782be15-e170-4777-a9d0-8cd5dbb80e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.368703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782be15-e170-4777-a9d0-8cd5dbb80e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.369121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcec04-fdf8-4309-91ea-9a26fccdcca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.369432 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782be15-e170-4777-a9d0-8cd5dbb80e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.370845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcec04-fdf8-4309-91ea-9a26fccdcca5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.371184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782be15-e170-4777-a9d0-8cd5dbb80e57 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.381622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 789/1589] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.384988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 801/1590] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.385155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcec04-fdf8-4309-91ea-9a26fccdcca5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.385261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782be15-e170-4777-a9d0-8cd5dbb80e57 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.393340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782be15-e170-4777-a9d0-8cd5dbb80e57 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.394511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcec04-fdf8-4309-91ea-9a26fccdcca5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.398522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c637e-bcd0-4183-a1c2-7bbaf97c06b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.398959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7166ad-023f-415a-bc85-43a36b1096ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.399211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c637e-bcd0-4183-a1c2-7bbaf97c06b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.399494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c637e-bcd0-4183-a1c2-7bbaf97c06b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.399597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7166ad-023f-415a-bc85-43a36b1096ac admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.399774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c637e-bcd0-4183-a1c2-7bbaf97c06b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.399879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7166ad-023f-415a-bc85-43a36b1096ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.400134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7166ad-023f-415a-bc85-43a36b1096ac admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.400498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c637e-bcd0-4183-a1c2-7bbaf97c06b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.400816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7166ad-023f-415a-bc85-43a36b1096ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.402239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c637e-bcd0-4183-a1c2-7bbaf97c06b6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.402898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7166ad-023f-415a-bc85-43a36b1096ac admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.413045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 802/1591] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.413653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c637e-bcd0-4183-a1c2-7bbaf97c06b6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.428096 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 790/1592] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:15 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:15.428274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c637e-bcd0-4183-a1c2-7bbaf97c06b6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.428404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7166ad-023f-415a-bc85-43a36b1096ac admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.430558 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d2519-30f7-40e0-8fe0-a15e082f8a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.431403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d2519-30f7-40e0-8fe0-a15e082f8a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.432201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d2519-30f7-40e0-8fe0-a15e082f8a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.432513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d2519-30f7-40e0-8fe0-a15e082f8a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.433563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d2519-30f7-40e0-8fe0-a15e082f8a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.435279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d2519-30f7-40e0-8fe0-a15e082f8a0f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.435500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7166ad-023f-415a-bc85-43a36b1096ac admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.440240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1e0d5-d055-40f5-8f0b-19be1aabc8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.440840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1e0d5-d055-40f5-8f0b-19be1aabc8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.441119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1e0d5-d055-40f5-8f0b-19be1aabc8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.441398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1e0d5-d055-40f5-8f0b-19be1aabc8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.442218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1e0d5-d055-40f5-8f0b-19be1aabc8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.443749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d1e0d5-d055-40f5-8f0b-19be1aabc8f7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.445346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 803/1593] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.446607 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d2519-30f7-40e0-8fe0-a15e082f8a0f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.455602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 791/1594] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.456365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d2519-30f7-40e0-8fe0-a15e082f8a0f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.456464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1e0d5-d055-40f5-8f0b-19be1aabc8f7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.461323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130554d5-6595-4f20-85d6-ea600cf8eef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.461978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130554d5-6595-4f20-85d6-ea600cf8eef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.462443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130554d5-6595-4f20-85d6-ea600cf8eef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.462707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130554d5-6595-4f20-85d6-ea600cf8eef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.463427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130554d5-6595-4f20-85d6-ea600cf8eef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.465068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130554d5-6595-4f20-85d6-ea600cf8eef3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.466433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1e0d5-d055-40f5-8f0b-19be1aabc8f7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.471470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c099b-4893-4660-afc8-1f72ceef4c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.472135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c099b-4893-4660-afc8-1f72ceef4c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.472430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c099b-4893-4660-afc8-1f72ceef4c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.472714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c099b-4893-4660-afc8-1f72ceef4c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.473440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c099b-4893-4660-afc8-1f72ceef4c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.475098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c099b-4893-4660-afc8-1f72ceef4c22 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.475960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 804/1595] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.476786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130554d5-6595-4f20-85d6-ea600cf8eef3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.486450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 792/1596] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.486820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130554d5-6595-4f20-85d6-ea600cf8eef3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.487637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c099b-4893-4660-afc8-1f72ceef4c22 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.494480 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884d471-533d-4c79-9046-6ff424ad1bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.495456 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884d471-533d-4c79-9046-6ff424ad1bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.496218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884d471-533d-4c79-9046-6ff424ad1bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.496218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884d471-533d-4c79-9046-6ff424ad1bdd admin admin] PATH_INFO: `/v3/projects/1e27652a0a2f4421aed586e23a86d0ae/users/14d3e513d8ce4efc87a359e04a77f05b/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.498513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c099b-4893-4660-afc8-1f72ceef4c22 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.503376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c815f9-4f04-4764-9ded-b770c2ef36b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.504034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c815f9-4f04-4764-9ded-b770c2ef36b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.504316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c815f9-4f04-4764-9ded-b770c2ef36b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.504593 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c815f9-4f04-4764-9ded-b770c2ef36b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.505312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c815f9-4f04-4764-9ded-b770c2ef36b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.506983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c815f9-4f04-4764-9ded-b770c2ef36b1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.511278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884d471-533d-4c79-9046-6ff424ad1bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e27652a0a2f4421aed586e23a86d0ae, user_id=14d3e513d8ce4efc87a359e04a77f05b, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.514101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884d471-533d-4c79-9046-6ff424ad1bdd admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.517009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 793/1597] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.517812 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c815f9-4f04-4764-9ded-b770c2ef36b1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.528492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c815f9-4f04-4764-9ded-b770c2ef36b1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.531209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 805/1598] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:15 2022] PUT /identity/v3/projects/1e27652a0a2f4421aed586e23a86d0ae/users/14d3e513d8ce4efc87a359e04a77f05b/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:15.532026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884d471-533d-4c79-9046-6ff424ad1bdd admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.537073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:15.547838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:15.590736 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f0aee-ce6a-445e-b9bc-cb07200835c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.591442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0aee-ce6a-445e-b9bc-cb07200835c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.591719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0aee-ce6a-445e-b9bc-cb07200835c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.591995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f0aee-ce6a-445e-b9bc-cb07200835c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.592868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f0aee-ce6a-445e-b9bc-cb07200835c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.594504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f0aee-ce6a-445e-b9bc-cb07200835c5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.601449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884d471-533d-4c79-9046-6ff424ad1bdd admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.604133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 794/1599] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.604965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f0aee-ce6a-445e-b9bc-cb07200835c5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.608968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c45551-4b7b-48d4-8560-be86dd426a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.609652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c45551-4b7b-48d4-8560-be86dd426a77 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.609935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c45551-4b7b-48d4-8560-be86dd426a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.610240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c45551-4b7b-48d4-8560-be86dd426a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.610974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c45551-4b7b-48d4-8560-be86dd426a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.612703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c45551-4b7b-48d4-8560-be86dd426a77 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.614180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f0aee-ce6a-445e-b9bc-cb07200835c5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.618686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a57f03-4238-4e07-bdaa-31ae3286049b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.619549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a57f03-4238-4e07-bdaa-31ae3286049b admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.619827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a57f03-4238-4e07-bdaa-31ae3286049b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.620105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a57f03-4238-4e07-bdaa-31ae3286049b admin admin] PATH_INFO: `/v3/projects/91f07f1f9f8c4be59e9bd3e0be4a1ea3/users/6bd54a561aa948888095879de89cd98b/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.626157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 806/1600] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:15.626157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c45551-4b7b-48d4-8560-be86dd426a77 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.633888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c45551-4b7b-48d4-8560-be86dd426a77 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.636372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a57f03-4238-4e07-bdaa-31ae3286049b admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f07f1f9f8c4be59e9bd3e0be4a1ea3, user_id=6bd54a561aa948888095879de89cd98b, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.638813 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c1e1e-ac8d-4b44-b569-a16c88f26702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.639093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a57f03-4238-4e07-bdaa-31ae3286049b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.639753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c1e1e-ac8d-4b44-b569-a16c88f26702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.640040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c1e1e-ac8d-4b44-b569-a16c88f26702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.640317 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c1e1e-ac8d-4b44-b569-a16c88f26702 admin admin] PATH_INFO: `/v3/projects/4cab4c10ddc745418a470e5d50b77bea/users/0c4a4ce86beb47c2b7910264a9238c8b/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.652793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 795/1601] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:15 2022] PUT /identity/v3/projects/91f07f1f9f8c4be59e9bd3e0be4a1ea3/users/6bd54a561aa948888095879de89cd98b/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:15.654862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a57f03-4238-4e07-bdaa-31ae3286049b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.656178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c1e1e-ac8d-4b44-b569-a16c88f26702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cab4c10ddc745418a470e5d50b77bea, user_id=0c4a4ce86beb47c2b7910264a9238c8b, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.658854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c1e1e-ac8d-4b44-b569-a16c88f26702 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.672068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 807/1602] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:15 2022] PUT /identity/v3/projects/4cab4c10ddc745418a470e5d50b77bea/users/0c4a4ce86beb47c2b7910264a9238c8b/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:15.672861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c1e1e-ac8d-4b44-b569-a16c88f26702 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.727902 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a57f03-4238-4e07-bdaa-31ae3286049b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.732776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53fb7b-ca80-46bf-b873-935d399d73b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.733690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53fb7b-ca80-46bf-b873-935d399d73b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.734002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53fb7b-ca80-46bf-b873-935d399d73b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.734297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c1e1e-ac8d-4b44-b569-a16c88f26702 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.734432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53fb7b-ca80-46bf-b873-935d399d73b9 admin admin] PATH_INFO: `/v3/projects/7eb9dd7f67cb48cebba2dabcd32a21d7/users/48e77af2b3a2409fa6d352b0d2674b85/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.738956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95d503-1630-427a-8f0b-1a467b6377e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.740084 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d503-1630-427a-8f0b-1a467b6377e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.740370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d503-1630-427a-8f0b-1a467b6377e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.740648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95d503-1630-427a-8f0b-1a467b6377e8 admin admin] PATH_INFO: `/v3/projects/6a8e35f84ffb489cbfd0d18d29451367/users/1e5a30a9e4044d158e9bd597c6282066/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.748701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53fb7b-ca80-46bf-b873-935d399d73b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eb9dd7f67cb48cebba2dabcd32a21d7, user_id=48e77af2b3a2409fa6d352b0d2674b85, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.751464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53fb7b-ca80-46bf-b873-935d399d73b9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.755567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95d503-1630-427a-8f0b-1a467b6377e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8e35f84ffb489cbfd0d18d29451367, user_id=1e5a30a9e4044d158e9bd597c6282066, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.758392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95d503-1630-427a-8f0b-1a467b6377e8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.767215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 796/1603] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:15 2022] PUT /identity/v3/projects/7eb9dd7f67cb48cebba2dabcd32a21d7/users/48e77af2b3a2409fa6d352b0d2674b85/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:15.768164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53fb7b-ca80-46bf-b873-935d399d73b9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.776125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 808/1604] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:15 2022] PUT /identity/v3/projects/6a8e35f84ffb489cbfd0d18d29451367/users/1e5a30a9e4044d158e9bd597c6282066/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:15.777707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646efb4-e6dc-4278-b2da-c690ad848c94 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.778053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646efb4-e6dc-4278-b2da-c690ad848c94 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.778434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646efb4-e6dc-4278-b2da-c690ad848c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.795893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3646efb4-e6dc-4278-b2da-c690ad848c94 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:15.797186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3646efb4-e6dc-4278-b2da-c690ad848c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:15.829723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53fb7b-ca80-46bf-b873-935d399d73b9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.838704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:15.857438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 809/1605] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:15.861618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646efb4-e6dc-4278-b2da-c690ad848c94 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.871478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646efb4-e6dc-4278-b2da-c690ad848c94 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:15.876047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c51a3-dec7-446d-ac4e-662ca009e7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.876965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c51a3-dec7-446d-ac4e-662ca009e7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.877247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c51a3-dec7-446d-ac4e-662ca009e7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.877530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c51a3-dec7-446d-ac4e-662ca009e7bc admin admin] PATH_INFO: `/v3/projects/bc0db5d97aee425dbfc522a22162a6b1/users/3170ca4f0dfb40768cef22a1be8a95e1/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.879789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:15.891291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f4047-2211-4d0c-ad0e-5b5ae72ad75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:15.892473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f4047-2211-4d0c-ad0e-5b5ae72ad75b admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.892763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f4047-2211-4d0c-ad0e-5b5ae72ad75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.893041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f4047-2211-4d0c-ad0e-5b5ae72ad75b admin admin] PATH_INFO: `/v3/projects/d40d4d8339dc408fb620ada67f2ed9e3/users/3019e93bb0bb42508d6eb90b9bd44f7e/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.893306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c51a3-dec7-446d-ac4e-662ca009e7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0db5d97aee425dbfc522a22162a6b1, user_id=3170ca4f0dfb40768cef22a1be8a95e1, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.896032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c51a3-dec7-446d-ac4e-662ca009e7bc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.909743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f4047-2211-4d0c-ad0e-5b5ae72ad75b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40d4d8339dc408fb620ada67f2ed9e3, user_id=3019e93bb0bb42508d6eb90b9bd44f7e, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:15.912867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f4047-2211-4d0c-ad0e-5b5ae72ad75b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:15.914180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 810/1606] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:15 2022] PUT /identity/v3/projects/bc0db5d97aee425dbfc522a22162a6b1/users/3170ca4f0dfb40768cef22a1be8a95e1/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:15.915153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c51a3-dec7-446d-ac4e-662ca009e7bc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:15.928492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 797/1607] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:15 2022] PUT /identity/v3/projects/d40d4d8339dc408fb620ada67f2ed9e3/users/3019e93bb0bb42508d6eb90b9bd44f7e/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:15.929869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc5d3e-059f-46b1-9e83-75c8b7b110c7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:15.930261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc5d3e-059f-46b1-9e83-75c8b7b110c7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:15.930542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc5d3e-059f-46b1-9e83-75c8b7b110c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:15.956686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6ffc5d3e-059f-46b1-9e83-75c8b7b110c7 None None] MFA Rules not processed for user `6bd54a561aa948888095879de89cd98b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:15.958159 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6ffc5d3e-059f-46b1-9e83-75c8b7b110c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:15.988248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c51a3-dec7-446d-ac4e-662ca009e7bc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:16.021663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 798/1608] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:16.023061 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862257-1263-4013-9885-13b74e8bfdc1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.023354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862257-1263-4013-9885-13b74e8bfdc1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.023635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862257-1263-4013-9885-13b74e8bfdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.041657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-48862257-1263-4013-9885-13b74e8bfdc1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:16.043268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-48862257-1263-4013-9885-13b74e8bfdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:16.043543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b884d-9d1f-42bd-86d5-925498370e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:16.044232 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b884d-9d1f-42bd-86d5-925498370e26 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.044517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b884d-9d1f-42bd-86d5-925498370e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.044796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b884d-9d1f-42bd-86d5-925498370e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.045538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b884d-9d1f-42bd-86d5-925498370e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:16.047403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b884d-9d1f-42bd-86d5-925498370e26 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:16.052155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 799/1609] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:16.055491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52301208-1984-490c-9745-9d1f3ccdbcc6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.055491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52301208-1984-490c-9745-9d1f3ccdbcc6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.055491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52301208-1984-490c-9745-9d1f3ccdbcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.061559 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 811/1610] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:16.062569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b884d-9d1f-42bd-86d5-925498370e26 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.072576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-52301208-1984-490c-9745-9d1f3ccdbcc6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:16.072748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b884d-9d1f-42bd-86d5-925498370e26 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:16.073790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-52301208-1984-490c-9745-9d1f3ccdbcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:16.077593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc0bfc-295d-4083-8826-2a3b107541da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:16.078299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0bfc-295d-4083-8826-2a3b107541da admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.078585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0bfc-295d-4083-8826-2a3b107541da admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.078869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0bfc-295d-4083-8826-2a3b107541da admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.079591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc0bfc-295d-4083-8826-2a3b107541da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:16.081203 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc0bfc-295d-4083-8826-2a3b107541da admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:16.081641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 800/1611] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:16.082300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52301208-1984-490c-9745-9d1f3ccdbcc6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.091701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 812/1612] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:16.092841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce58d23-c193-41cc-8b92-1550cf3557dd None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.093135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce58d23-c193-41cc-8b92-1550cf3557dd None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.093414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce58d23-c193-41cc-8b92-1550cf3557dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.118875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bce58d23-c193-41cc-8b92-1550cf3557dd None None] MFA Rules not processed for user `3170ca4f0dfb40768cef22a1be8a95e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:16.120081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bce58d23-c193-41cc-8b92-1550cf3557dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:16.142450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52301208-1984-490c-9745-9d1f3ccdbcc6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:16.146050 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f478893-1e6a-4eb8-b3a6-9b3754b95f42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:16.146759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f478893-1e6a-4eb8-b3a6-9b3754b95f42 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.147046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f478893-1e6a-4eb8-b3a6-9b3754b95f42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.147327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f478893-1e6a-4eb8-b3a6-9b3754b95f42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.156715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f478893-1e6a-4eb8-b3a6-9b3754b95f42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:16.158224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f478893-1e6a-4eb8-b3a6-9b3754b95f42 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:16.167667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 801/1613] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:16.168876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723bb2d-bb23-44c1-b46b-babed517f034 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.169165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723bb2d-bb23-44c1-b46b-babed517f034 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.169443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723bb2d-bb23-44c1-b46b-babed517f034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.185741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 813/1614] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:16.191910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce58d23-c193-41cc-8b92-1550cf3557dd None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.203741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:16.207572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7723bb2d-bb23-44c1-b46b-babed517f034 None None] MFA Rules not processed for user `3019e93bb0bb42508d6eb90b9bd44f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:16.208818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce58d23-c193-41cc-8b92-1550cf3557dd None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:16.208947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7723bb2d-bb23-44c1-b46b-babed517f034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:16.216747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:16.222847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a7202-dc3d-41eb-968e-9e57adcb6520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:16.223616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a7202-dc3d-41eb-968e-9e57adcb6520 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.223893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a7202-dc3d-41eb-968e-9e57adcb6520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.224146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a7202-dc3d-41eb-968e-9e57adcb6520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.224880 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a7202-dc3d-41eb-968e-9e57adcb6520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:16.226933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a7202-dc3d-41eb-968e-9e57adcb6520 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:16.270653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 814/1615] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:16 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:16.271921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a7202-dc3d-41eb-968e-9e57adcb6520 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.295529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 802/1616] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:16.296615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723bb2d-bb23-44c1-b46b-babed517f034 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.310206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723bb2d-bb23-44c1-b46b-babed517f034 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:16.350539 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a7202-dc3d-41eb-968e-9e57adcb6520 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:16.357294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9404691-8e5d-410e-8741-3226b56cef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:16.359153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9404691-8e5d-410e-8741-3226b56cef69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.359481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9404691-8e5d-410e-8741-3226b56cef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.359730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9404691-8e5d-410e-8741-3226b56cef69 admin admin] PATH_INFO: `/v3/projects/9f6fbcb132a94013b342b382e4e2e9ab/users/78b847b86a4b41229915f4b6441b07f2/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.369767 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f9d38-1e0d-43c6-a31b-65ac5ad367ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:16.371476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f9d38-1e0d-43c6-a31b-65ac5ad367ee service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.371960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f9d38-1e0d-43c6-a31b-65ac5ad367ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.372432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f9d38-1e0d-43c6-a31b-65ac5ad367ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.376990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9404691-8e5d-410e-8741-3226b56cef69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f6fbcb132a94013b342b382e4e2e9ab, user_id=78b847b86a4b41229915f4b6441b07f2, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:16.379857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9404691-8e5d-410e-8741-3226b56cef69 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:16.386098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f9d38-1e0d-43c6-a31b-65ac5ad367ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:16.388058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f9d38-1e0d-43c6-a31b-65ac5ad367ee service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:16.404642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 815/1617] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:16 2022] PUT /identity/v3/projects/9f6fbcb132a94013b342b382e4e2e9ab/users/78b847b86a4b41229915f4b6441b07f2/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:16.404642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9404691-8e5d-410e-8741-3226b56cef69 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.405095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 803/1618] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:16.405095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f9d38-1e0d-43c6-a31b-65ac5ad367ee service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.410691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f9d38-1e0d-43c6-a31b-65ac5ad367ee service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:16.475409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9404691-8e5d-410e-8741-3226b56cef69 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:16.477837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445ddbd-75cc-4994-b55b-8b4a5e2e73c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:16.478783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445ddbd-75cc-4994-b55b-8b4a5e2e73c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.479062 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445ddbd-75cc-4994-b55b-8b4a5e2e73c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.479337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445ddbd-75cc-4994-b55b-8b4a5e2e73c6 admin admin] PATH_INFO: `/v3/projects/6a8e35f84ffb489cbfd0d18d29451367/users/1e5a30a9e4044d158e9bd597c6282066/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.480245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54075a6-4c09-4e4e-b702-fe6672794eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:16.480961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54075a6-4c09-4e4e-b702-fe6672794eba service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.481309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54075a6-4c09-4e4e-b702-fe6672794eba service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.481585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54075a6-4c09-4e4e-b702-fe6672794eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.483084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445ddbd-75cc-4994-b55b-8b4a5e2e73c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8e35f84ffb489cbfd0d18d29451367, user_id=1e5a30a9e4044d158e9bd597c6282066, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:16.485748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445ddbd-75cc-4994-b55b-8b4a5e2e73c6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:16.492079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54075a6-4c09-4e4e-b702-fe6672794eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:16.493848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54075a6-4c09-4e4e-b702-fe6672794eba service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:16.498121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 804/1619] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:16 2022] PUT /identity/v3/projects/6a8e35f84ffb489cbfd0d18d29451367/users/1e5a30a9e4044d158e9bd597c6282066/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:16.499954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f8bfd-cfc6-4f03-b0e7-2d367b318369 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.499954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f8bfd-cfc6-4f03-b0e7-2d367b318369 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.500161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f8bfd-cfc6-4f03-b0e7-2d367b318369 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.501483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 805/1620] 10.222.0.33 () {54 vars in 961 bytes} [Mon Jun 20 11:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:16.505522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 816/1621] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:16.506524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54075a6-4c09-4e4e-b702-fe6672794eba service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.506713 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f8bfd-cfc6-4f03-b0e7-2d367b318369 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.559902 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:16.568704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54075a6-4c09-4e4e-b702-fe6672794eba service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:16.591058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f8bfd-cfc6-4f03-b0e7-2d367b318369 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:16.599263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed5110-4c9b-4746-ac08-66333b8a8f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:16.600289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed5110-4c9b-4746-ac08-66333b8a8f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.600600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:16.600777 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed5110-4c9b-4746-ac08-66333b8a8f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.600926 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed5110-4c9b-4746-ac08-66333b8a8f9e admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.601699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed5110-4c9b-4746-ac08-66333b8a8f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:16.603655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed5110-4c9b-4746-ac08-66333b8a8f9e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:16.627440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 806/1622] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:16 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:16.631214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e302e34-ed12-489a-99c0-a7fbaf2533f4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.632347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e302e34-ed12-489a-99c0-a7fbaf2533f4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.632347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e302e34-ed12-489a-99c0-a7fbaf2533f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.637628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c7e5c-01a5-4004-846d-5cc472bf1da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:16.638515 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c7e5c-01a5-4004-846d-5cc472bf1da2 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.639047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c7e5c-01a5-4004-846d-5cc472bf1da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.639340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c7e5c-01a5-4004-846d-5cc472bf1da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.665781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0e302e34-ed12-489a-99c0-a7fbaf2533f4 None None] MFA Rules not processed for user `78b847b86a4b41229915f4b6441b07f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:16.667079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0e302e34-ed12-489a-99c0-a7fbaf2533f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:16.709535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c7e5c-01a5-4004-846d-5cc472bf1da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:16.711533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c7e5c-01a5-4004-846d-5cc472bf1da2 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:16.722948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 817/1623] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:16.724768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d49ea-90c7-4645-8a1a-fa26a857cb7c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.725064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d49ea-90c7-4645-8a1a-fa26a857cb7c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.725945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d49ea-90c7-4645-8a1a-fa26a857cb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.758238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4e0d49ea-90c7-4645-8a1a-fa26a857cb7c None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:16.760263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4e0d49ea-90c7-4645-8a1a-fa26a857cb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:16.783650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 807/1624] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:16.786553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4824c7-4114-4ecf-adee-3a47f31829be None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:16.788701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4824c7-4114-4ecf-adee-3a47f31829be None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:16.789099 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4824c7-4114-4ecf-adee-3a47f31829be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:16.867520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dd4824c7-4114-4ecf-adee-3a47f31829be None None] MFA Rules not processed for user `1e5a30a9e4044d158e9bd597c6282066`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:16.869921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dd4824c7-4114-4ecf-adee-3a47f31829be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:16.959379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 818/1625] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Jun 20 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:16.961793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d49ea-90c7-4645-8a1a-fa26a857cb7c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.983893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 808/1626] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:16.985407 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4824c7-4114-4ecf-adee-3a47f31829be None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:16.986106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d49ea-90c7-4645-8a1a-fa26a857cb7c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:17.011096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e7635-cf31-4c2b-a8b9-1f29e596c632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:17.014022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e7635-cf31-4c2b-a8b9-1f29e596c632 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.014697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e7635-cf31-4c2b-a8b9-1f29e596c632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.016085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e7635-cf31-4c2b-a8b9-1f29e596c632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.019590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e7635-cf31-4c2b-a8b9-1f29e596c632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:17.024527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e7635-cf31-4c2b-a8b9-1f29e596c632 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:17.055385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 819/1627] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:17.055385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e7635-cf31-4c2b-a8b9-1f29e596c632 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:17.057037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4824c7-4114-4ecf-adee-3a47f31829be None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:17.065083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35939fc5-6162-42d8-91e7-4a82845dbbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:17.065850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35939fc5-6162-42d8-91e7-4a82845dbbff admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.066187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35939fc5-6162-42d8-91e7-4a82845dbbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.066744 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35939fc5-6162-42d8-91e7-4a82845dbbff admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.067629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35939fc5-6162-42d8-91e7-4a82845dbbff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:17.071098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35939fc5-6162-42d8-91e7-4a82845dbbff admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:17.088913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e7635-cf31-4c2b-a8b9-1f29e596c632 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:17.110280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb790f-b664-4ec6-81a9-77d614ed96ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:17.111132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb790f-b664-4ec6-81a9-77d614ed96ef service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.111926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb790f-b664-4ec6-81a9-77d614ed96ef service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.112209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb790f-b664-4ec6-81a9-77d614ed96ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.131314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 809/1628] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:16 2022] POST /identity/v3/projects => generated 403 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:17.132856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35939fc5-6162-42d8-91e7-4a82845dbbff admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:17.177138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:17.242917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35939fc5-6162-42d8-91e7-4a82845dbbff admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:17.250385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb790f-b664-4ec6-81a9-77d614ed96ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:17.258385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb790f-b664-4ec6-81a9-77d614ed96ef service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:17.259265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816898a4-9fcb-4c58-a693-d320c5f8c866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:17.260030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816898a4-9fcb-4c58-a693-d320c5f8c866 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.260326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816898a4-9fcb-4c58-a693-d320c5f8c866 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.260608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816898a4-9fcb-4c58-a693-d320c5f8c866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.266231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:17.284223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 820/1629] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:17.285085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb790f-b664-4ec6-81a9-77d614ed96ef service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:17.308542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb790f-b664-4ec6-81a9-77d614ed96ef service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:17.415131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816898a4-9fcb-4c58-a693-d320c5f8c866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:17.416624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816898a4-9fcb-4c58-a693-d320c5f8c866 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:17.438868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 810/1630] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:17.440422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816898a4-9fcb-4c58-a693-d320c5f8c866 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:17.451303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e04f1-b10c-46b5-959c-89014bd9b4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:17.453311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e04f1-b10c-46b5-959c-89014bd9b4fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.454552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e04f1-b10c-46b5-959c-89014bd9b4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.454859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e04f1-b10c-46b5-959c-89014bd9b4fa admin admin] PATH_INFO: `/v3/projects/013a9ec3d79745be8736fe2cb7d2a9c1/users/aa5c0b89e3d84736b1478610da6c0e57/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.488334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e04f1-b10c-46b5-959c-89014bd9b4fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=013a9ec3d79745be8736fe2cb7d2a9c1, user_id=aa5c0b89e3d84736b1478610da6c0e57, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:17.495559 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e04f1-b10c-46b5-959c-89014bd9b4fa admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:17.523847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 821/1631] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:17 2022] PUT /identity/v3/projects/013a9ec3d79745be8736fe2cb7d2a9c1/users/aa5c0b89e3d84736b1478610da6c0e57/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:17.532071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e04f1-b10c-46b5-959c-89014bd9b4fa admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:17.585180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816898a4-9fcb-4c58-a693-d320c5f8c866 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:17.659089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e04f1-b10c-46b5-959c-89014bd9b4fa admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:17.684637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cccc8-ee5a-4ee6-88c9-2c15a3bd1006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:17.686281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d99303-f569-4846-a665-a3eaae99853e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:17.687140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d99303-f569-4846-a665-a3eaae99853e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.687261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d99303-f569-4846-a665-a3eaae99853e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.687546 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d99303-f569-4846-a665-a3eaae99853e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.687656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cccc8-ee5a-4ee6-88c9-2c15a3bd1006 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.688296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d99303-f569-4846-a665-a3eaae99853e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:17.688680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cccc8-ee5a-4ee6-88c9-2c15a3bd1006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.689910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d99303-f569-4846-a665-a3eaae99853e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:17.690271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cccc8-ee5a-4ee6-88c9-2c15a3bd1006 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.691147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cccc8-ee5a-4ee6-88c9-2c15a3bd1006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:17.696968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cccc8-ee5a-4ee6-88c9-2c15a3bd1006 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:17.716422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 822/1632] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:17.716725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d99303-f569-4846-a665-a3eaae99853e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:17.742060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 811/1633] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:17 2022] POST /identity/v3/users => generated 308 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:17.747382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cccc8-ee5a-4ee6-88c9-2c15a3bd1006 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:17.754423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d99303-f569-4846-a665-a3eaae99853e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:17.767546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cccc8-ee5a-4ee6-88c9-2c15a3bd1006 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:17.772260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fedd6b-96de-467b-994b-b475f1ad003b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:17.773185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fedd6b-96de-467b-994b-b475f1ad003b admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.773471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fedd6b-96de-467b-994b-b475f1ad003b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.773750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fedd6b-96de-467b-994b-b475f1ad003b admin admin] PATH_INFO: `/v3/projects/013a9ec3d79745be8736fe2cb7d2a9c1/users/aa5c0b89e3d84736b1478610da6c0e57/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.779985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fedd6b-96de-467b-994b-b475f1ad003b admin admin] RBAC: Authorizing `identity:create_grant(project_id=013a9ec3d79745be8736fe2cb7d2a9c1, user_id=aa5c0b89e3d84736b1478610da6c0e57, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:17.790881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a6a81-a392-4014-9aa5-f13002c2022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:17.791597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a6a81-a392-4014-9aa5-f13002c2022d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.793355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a6a81-a392-4014-9aa5-f13002c2022d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.793355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a6a81-a392-4014-9aa5-f13002c2022d admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.793355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a6a81-a392-4014-9aa5-f13002c2022d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:17.793965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fedd6b-96de-467b-994b-b475f1ad003b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:17.794703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a6a81-a392-4014-9aa5-f13002c2022d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:17.808187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:17.825271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 812/1634] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:04:17.829563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 823/1635] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:17 2022] PUT /identity/v3/projects/013a9ec3d79745be8736fe2cb7d2a9c1/users/aa5c0b89e3d84736b1478610da6c0e57/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:17.830138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fedd6b-96de-467b-994b-b475f1ad003b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:17.830894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89f1ab-256c-472f-9f74-dca6378b0667 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.831480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89f1ab-256c-472f-9f74-dca6378b0667 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.831992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89f1ab-256c-472f-9f74-dca6378b0667 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.839208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 813/1636] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jun 20 11:04:17 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:17.841038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4581a1-752d-40a6-bb76-8eb1239f0ea7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:17.841333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4581a1-752d-40a6-bb76-8eb1239f0ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:17.842337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4581a1-752d-40a6-bb76-8eb1239f0ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:17.900997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ac4581a1-752d-40a6-bb76-8eb1239f0ea7 None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:17.902478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ac4581a1-752d-40a6-bb76-8eb1239f0ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:17.984304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fedd6b-96de-467b-994b-b475f1ad003b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:18.002938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c9abe-a5ac-44db-901a-7463a27d70ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:18.004548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c9abe-a5ac-44db-901a-7463a27d70ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:18.004839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c9abe-a5ac-44db-901a-7463a27d70ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:18.006913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c9abe-a5ac-44db-901a-7463a27d70ca admin admin] PATH_INFO: `/v3/projects/dfd2898c1e2740b3985343d7daf1361c/users/f120fb9db75140b49cfd028e800c994c/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:18.024569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:18.053051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c9abe-a5ac-44db-901a-7463a27d70ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd2898c1e2740b3985343d7daf1361c, user_id=f120fb9db75140b49cfd028e800c994c, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:18.067945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c9abe-a5ac-44db-901a-7463a27d70ca admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:18.078368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 814/1637] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:04:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:18.079954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f917cdc-1ba1-48cb-bd71-a34ff69ec711 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:18.080634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f917cdc-1ba1-48cb-bd71-a34ff69ec711 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:18.080756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f917cdc-1ba1-48cb-bd71-a34ff69ec711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:18.098270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 824/1638] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:17 2022] PUT /identity/v3/projects/dfd2898c1e2740b3985343d7daf1361c/users/f120fb9db75140b49cfd028e800c994c/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:18.102721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c9abe-a5ac-44db-901a-7463a27d70ca admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:18.145427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1f917cdc-1ba1-48cb-bd71-a34ff69ec711 None None] MFA Rules not processed for user `aa5c0b89e3d84736b1478610da6c0e57`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:18.155699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1f917cdc-1ba1-48cb-bd71-a34ff69ec711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:18.177435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c9abe-a5ac-44db-901a-7463a27d70ca admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:18.190736 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3bd81-ad6f-4ce8-9b91-489746491640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:18.192571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3bd81-ad6f-4ce8-9b91-489746491640 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:18.193461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3bd81-ad6f-4ce8-9b91-489746491640 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:18.193758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3bd81-ad6f-4ce8-9b91-489746491640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:18.265223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 815/1639] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:18.265223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f917cdc-1ba1-48cb-bd71-a34ff69ec711 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:18.296201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3bd81-ad6f-4ce8-9b91-489746491640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:18.297811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3bd81-ad6f-4ce8-9b91-489746491640 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:18.310818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 825/1640] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:18.311485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3bd81-ad6f-4ce8-9b91-489746491640 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:18.332097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3bd81-ad6f-4ce8-9b91-489746491640 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:18.338739 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bde7a-5c62-4183-a946-c6af3e85db82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:18.341352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bde7a-5c62-4183-a946-c6af3e85db82 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:18.341622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bde7a-5c62-4183-a946-c6af3e85db82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:18.341901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bde7a-5c62-4183-a946-c6af3e85db82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:18.357639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bde7a-5c62-4183-a946-c6af3e85db82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:18.359341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bde7a-5c62-4183-a946-c6af3e85db82 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:18.374610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 826/1641] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:18.415859 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f917cdc-1ba1-48cb-bd71-a34ff69ec711 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:18.424459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b95ed-80a7-4d67-afd6-4dd56cd3ec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:18.425223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b95ed-80a7-4d67-afd6-4dd56cd3ec26 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:18.428895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b95ed-80a7-4d67-afd6-4dd56cd3ec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:18.429231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b95ed-80a7-4d67-afd6-4dd56cd3ec26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:18.432441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b95ed-80a7-4d67-afd6-4dd56cd3ec26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:18.448184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b95ed-80a7-4d67-afd6-4dd56cd3ec26 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:18.468269 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 816/1642] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:18.475812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bde7a-5c62-4183-a946-c6af3e85db82 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:18.496455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bde7a-5c62-4183-a946-c6af3e85db82 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:18.504937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0849d-98d4-4696-99bf-8b9c9059d468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:18.506444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0849d-98d4-4696-99bf-8b9c9059d468 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:18.507073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0849d-98d4-4696-99bf-8b9c9059d468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:18.507663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0849d-98d4-4696-99bf-8b9c9059d468 admin admin] PATH_INFO: `/v3/projects/dfd2898c1e2740b3985343d7daf1361c/users/f120fb9db75140b49cfd028e800c994c/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:18.510075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0849d-98d4-4696-99bf-8b9c9059d468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd2898c1e2740b3985343d7daf1361c, user_id=f120fb9db75140b49cfd028e800c994c, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:18.513094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0849d-98d4-4696-99bf-8b9c9059d468 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:18.541329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 827/1643] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:18 2022] PUT /identity/v3/projects/dfd2898c1e2740b3985343d7daf1361c/users/f120fb9db75140b49cfd028e800c994c/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:18.543478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6baae4-765c-4544-aa12-d8104eb6b597 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:18.543855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6baae4-765c-4544-aa12-d8104eb6b597 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:18.544101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6baae4-765c-4544-aa12-d8104eb6b597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:18.605660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:18.613873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3c6baae4-765c-4544-aa12-d8104eb6b597 None None] MFA Rules not processed for user `f120fb9db75140b49cfd028e800c994c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:18.613873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3c6baae4-765c-4544-aa12-d8104eb6b597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:18.627783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0849d-98d4-4696-99bf-8b9c9059d468 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:18.737343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0849d-98d4-4696-99bf-8b9c9059d468 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:18.743852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b69c3f-1dc2-4207-a8df-12c6db05512f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:18.744896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b69c3f-1dc2-4207-a8df-12c6db05512f service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:18.745570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b69c3f-1dc2-4207-a8df-12c6db05512f service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:18.745750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b69c3f-1dc2-4207-a8df-12c6db05512f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:18.771955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:18.809544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 817/1644] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:18 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:04:18.863269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b69c3f-1dc2-4207-a8df-12c6db05512f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:18.864971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b69c3f-1dc2-4207-a8df-12c6db05512f service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:18.886708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 828/1645] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:04:19.126712 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6baae4-765c-4544-aa12-d8104eb6b597 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:19.139576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6baae4-765c-4544-aa12-d8104eb6b597 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:19.171515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585e59a-7a70-455b-917a-fcbaf096f09a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:19.172208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585e59a-7a70-455b-917a-fcbaf096f09a service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:19.172489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585e59a-7a70-455b-917a-fcbaf096f09a service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:19.191101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585e59a-7a70-455b-917a-fcbaf096f09a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:19.228867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585e59a-7a70-455b-917a-fcbaf096f09a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:19.237767 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585e59a-7a70-455b-917a-fcbaf096f09a service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:19.252632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b69c3f-1dc2-4207-a8df-12c6db05512f service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:19.299634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 818/1646] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:19 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:19.437730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b69c3f-1dc2-4207-a8df-12c6db05512f service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:19.451007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:19.452105 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:19.452393 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:19.452674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] PATH_INFO: `/v3/users/16aecb7081804b929dd581903c2d6166` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:19.454815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16aecb7081804b929dd581903c2d6166)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:19.455866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:04:19.455866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Jun 20 11:04:19.455866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: Jun 20 11:04:19.458994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:19.713057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585e59a-7a70-455b-917a-fcbaf096f09a service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:19.715730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16aecb7081804b929dd581903c2d6166'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:19.716426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585e59a-7a70-455b-917a-fcbaf096f09a service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:19.727460 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:19.730792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:19.732419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:19.732715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] PATH_INFO: `/v3/users/78b847b86a4b41229915f4b6441b07f2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:19.735604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16aecb7081804b929dd581903c2d6166'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:19.735767 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b847b86a4b41229915f4b6441b07f2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:19.736289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16aecb7081804b929dd581903c2d6166'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:19.737658 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16aecb7081804b929dd581903c2d6166'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:19.743689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:19.841938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c7f2de9f-3deb-45f6-9193-0b9df7ca00b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16aecb7081804b929dd581903c2d6166'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:19.874438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b847b86a4b41229915f4b6441b07f2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:19.875071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b847b86a4b41229915f4b6441b07f2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:19.889550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 829/1647] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:19 2022] DELETE /identity/v3/users/16aecb7081804b929dd581903c2d6166 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:19.896477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072e56e-3579-4a8a-bd74-25dca16fbf2f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:19.898337 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072e56e-3579-4a8a-bd74-25dca16fbf2f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:19.898874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b847b86a4b41229915f4b6441b07f2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:19.899524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b847b86a4b41229915f4b6441b07f2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:19.899960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072e56e-3579-4a8a-bd74-25dca16fbf2f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:19.905584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 830/1648] 10.222.0.33 () {54 vars in 961 bytes} [Mon Jun 20 11:04:19 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:19.909797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f6cab-904e-4a3c-972f-7c6d95da9bb0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:19.909797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f6cab-904e-4a3c-972f-7c6d95da9bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:19.909797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f6cab-904e-4a3c-972f-7c6d95da9bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:19.928655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-32f2281d-01df-47fd-9123-6d747aa8b414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b847b86a4b41229915f4b6441b07f2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:19.949149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-254f6cab-904e-4a3c-972f-7c6d95da9bb0 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:19.950698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-254f6cab-904e-4a3c-972f-7c6d95da9bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:19.979051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 819/1649] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:19 2022] DELETE /identity/v3/users/78b847b86a4b41229915f4b6441b07f2 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:04:19.980671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7572b2a-fd17-4ea1-8bda-19cd8ad5a8d3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:19.980967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7572b2a-fd17-4ea1-8bda-19cd8ad5a8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:19.981248 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7572b2a-fd17-4ea1-8bda-19cd8ad5a8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:19.993330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:20.024833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e7572b2a-fd17-4ea1-8bda-19cd8ad5a8d3 None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:20.027608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e7572b2a-fd17-4ea1-8bda-19cd8ad5a8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:20.050423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 831/1650] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:20.052620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2e50-f54a-494a-87d0-694a6d545672 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:20.053391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2e50-f54a-494a-87d0-694a6d545672 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:20.054147 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2e50-f54a-494a-87d0-694a6d545672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:20.086236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7dde2e50-f54a-494a-87d0-694a6d545672 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:20.088829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7dde2e50-f54a-494a-87d0-694a6d545672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:20.148206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:20.185065 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 820/1651] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Jun 20 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:20.191619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7572b2a-fd17-4ea1-8bda-19cd8ad5a8d3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:20.196687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e7572b2a-fd17-4ea1-8bda-19cd8ad5a8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:20.224120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 832/1652] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:20.226470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7d66c-6961-482a-b6df-d1207d5b4064 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:20.226763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7d66c-6961-482a-b6df-d1207d5b4064 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:20.227077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7d66c-6961-482a-b6df-d1207d5b4064 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:20.236081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 833/1653] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jun 20 11:04:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:20.237529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b2c5d-94d7-4534-a5dd-6e8ac95079f7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:20.237814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b2c5d-94d7-4534-a5dd-6e8ac95079f7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:20.238139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b2c5d-94d7-4534-a5dd-6e8ac95079f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:20.291309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-922b2c5d-94d7-4534-a5dd-6e8ac95079f7 None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:20.297825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-922b2c5d-94d7-4534-a5dd-6e8ac95079f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:20.333742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 834/1654] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:04:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:20.347990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b2c5d-94d7-4534-a5dd-6e8ac95079f7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:20.359104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-922b2c5d-94d7-4534-a5dd-6e8ac95079f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:20.397688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7572b2a-fd17-4ea1-8bda-19cd8ad5a8d3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:20.415696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b549def-46ab-49e4-933b-a758740234ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:20.416395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b549def-46ab-49e4-933b-a758740234ec service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:20.420451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b549def-46ab-49e4-933b-a758740234ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:20.420805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b549def-46ab-49e4-933b-a758740234ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:20.462073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b549def-46ab-49e4-933b-a758740234ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:20.463819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b549def-46ab-49e4-933b-a758740234ec service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:20.500880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 821/1655] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:20.513965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b549def-46ab-49e4-933b-a758740234ec service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:20.524261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b549def-46ab-49e4-933b-a758740234ec service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:20.534619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3ea5a-cebc-49e8-a2d0-19f89d293417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:20.535319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3ea5a-cebc-49e8-a2d0-19f89d293417 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:20.535608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3ea5a-cebc-49e8-a2d0-19f89d293417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:20.535891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3ea5a-cebc-49e8-a2d0-19f89d293417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:20.540856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b2c5d-94d7-4534-a5dd-6e8ac95079f7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:20.551359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778d0a2-4ae1-46b9-a432-4a254b082d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:20.551792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778d0a2-4ae1-46b9-a432-4a254b082d0f service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:20.552063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778d0a2-4ae1-46b9-a432-4a254b082d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:20.552331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778d0a2-4ae1-46b9-a432-4a254b082d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:20.556511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3ea5a-cebc-49e8-a2d0-19f89d293417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:20.566988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a3ea5a-cebc-49e8-a2d0-19f89d293417 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:20.585449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778d0a2-4ae1-46b9-a432-4a254b082d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:20.587510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778d0a2-4ae1-46b9-a432-4a254b082d0f service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:20.595705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 822/1656] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:04:20.602791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 835/1657] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:21.803688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3ea5a-cebc-49e8-a2d0-19f89d293417 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:21.805194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-78a3ea5a-cebc-49e8-a2d0-19f89d293417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:21.837379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a3ea5a-cebc-49e8-a2d0-19f89d293417 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:21.860630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:21.861530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:21.861963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:21.862436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] PATH_INFO: `/v3/projects/44ee128f927f4dc685c1a9a4bd0d6b55` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:21.871147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44ee128f927f4dc685c1a9a4bd0d6b55)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:21.872483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: WARNING py.warnings [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 11:04:21.872483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: warnings.warn(msg) Jun 20 11:04:21.872483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: Jun 20 11:04:21.876336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:22.099395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44ee128f927f4dc685c1a9a4bd0d6b55', 'request_id': 'req-23b4635d-1649-47f8-8488-434cdfc85ef3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.111604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44ee128f927f4dc685c1a9a4bd0d6b55', 'request_id': 'req-23b4635d-1649-47f8-8488-434cdfc85ef3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.135468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44ee128f927f4dc685c1a9a4bd0d6b55', 'request_id': 'req-23b4635d-1649-47f8-8488-434cdfc85ef3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.184796 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] The token cache is being invalidate because project 44ee128f927f4dc685c1a9a4bd0d6b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:22.185250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.186316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.194674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 823/1658] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:21 2022] DELETE /identity/v3/projects/44ee128f927f4dc685c1a9a4bd0d6b55 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:22.199377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778d0a2-4ae1-46b9-a432-4a254b082d0f service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:22.213589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778d0a2-4ae1-46b9-a432-4a254b082d0f service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:22.313865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:22.315899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:22.383731 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:22.384812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:22.387756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:22.388053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] PATH_INFO: `/v3/users/1e5a30a9e4044d158e9bd597c6282066` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:22.393566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e5a30a9e4044d158e9bd597c6282066)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:22.399644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:22.439952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4635d-1649-47f8-8488-434cdfc85ef3 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:22.446119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:22.453297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:22.455246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:22.455725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:22.456310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] PATH_INFO: `/v3/projects/9f6fbcb132a94013b342b382e4e2e9ab` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:22.463154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f6fbcb132a94013b342b382e4e2e9ab)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:22.463154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:22.469194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:22.555847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e5a30a9e4044d158e9bd597c6282066'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.595448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e5a30a9e4044d158e9bd597c6282066'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.596379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e5a30a9e4044d158e9bd597c6282066'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.600017 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f6fbcb132a94013b342b382e4e2e9ab', 'request_id': 'req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.601300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e5a30a9e4044d158e9bd597c6282066'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.621316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f6fbcb132a94013b342b382e4e2e9ab', 'request_id': 'req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.636194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f6fbcb132a94013b342b382e4e2e9ab', 'request_id': 'req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.677466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] The token cache is being invalidate because project 9f6fbcb132a94013b342b382e4e2e9ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:22.681678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.683420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.688597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e5a30a9e4044d158e9bd597c6282066'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:22.689466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 824/1659] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:22 2022] DELETE /identity/v3/projects/9f6fbcb132a94013b342b382e4e2e9ab => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:22.695068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:22.699615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:22.768462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 836/1660] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:22 2022] DELETE /identity/v3/users/1e5a30a9e4044d158e9bd597c6282066 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:22.851808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f87ea9-03b6-4bf4-bbc0-43c3679c204d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:22.951284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:22.952578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:22.954129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:22.954581 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] PATH_INFO: `/v3/users/f120fb9db75140b49cfd028e800c994c` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:22.956325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f120fb9db75140b49cfd028e800c994c)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:22.960187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:23.052579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:23.066606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f120fb9db75140b49cfd028e800c994c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:23.068119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f120fb9db75140b49cfd028e800c994c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:23.086381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f120fb9db75140b49cfd028e800c994c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:23.087391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f120fb9db75140b49cfd028e800c994c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:23.116613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-314a6169-c429-4a19-8388-9b009e7d02dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f120fb9db75140b49cfd028e800c994c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:23.161854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 825/1661] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:22 2022] DELETE /identity/v3/users/f120fb9db75140b49cfd028e800c994c => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:23.279974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:23.284412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:23.341373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d4051-f5a0-4d1e-aa75-60d3594c116d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:23.556010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:23.558562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:23.559042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:23.563544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] PATH_INFO: `/v3/users/3170ca4f0dfb40768cef22a1be8a95e1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:23.566552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3170ca4f0dfb40768cef22a1be8a95e1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:23.574812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:23.715805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3170ca4f0dfb40768cef22a1be8a95e1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:23.746624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3170ca4f0dfb40768cef22a1be8a95e1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:23.747428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3170ca4f0dfb40768cef22a1be8a95e1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:23.748475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3170ca4f0dfb40768cef22a1be8a95e1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:23.784448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3170ca4f0dfb40768cef22a1be8a95e1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:23.809637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 837/1662] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:23 2022] DELETE /identity/v3/users/3170ca4f0dfb40768cef22a1be8a95e1 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:23.813906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79eec1-2291-4275-9d87-d3beba942913 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:23.814461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79eec1-2291-4275-9d87-d3beba942913 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:23.814936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79eec1-2291-4275-9d87-d3beba942913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:23.857816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bf79eec1-2291-4275-9d87-d3beba942913 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:23.860044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bf79eec1-2291-4275-9d87-d3beba942913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:23.991167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 826/1663] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:24.050738 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:24.052291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:24.076800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:24.181023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d406b-8dce-425c-9860-f1ed90d42493 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:24.195189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32cd0-c643-4622-a771-8d602288d901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:24.198074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32cd0-c643-4622-a771-8d602288d901 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:24.198435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32cd0-c643-4622-a771-8d602288d901 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:24.198715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c32cd0-c643-4622-a771-8d602288d901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:24.221158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c32cd0-c643-4622-a771-8d602288d901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:24.223657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c32cd0-c643-4622-a771-8d602288d901 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:24.260495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 838/1664] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:25.286443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79eec1-2291-4275-9d87-d3beba942913 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:25.288984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bf79eec1-2291-4275-9d87-d3beba942913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:25.314263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79eec1-2291-4275-9d87-d3beba942913 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:25.319580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:25.320494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:25.321217 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:25.321502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] PATH_INFO: `/v3/projects/dfd2898c1e2740b3985343d7daf1361c` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:25.322786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd2898c1e2740b3985343d7daf1361c)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:25.324836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:25.330978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32cd0-c643-4622-a771-8d602288d901 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:25.332431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-13c32cd0-c643-4622-a771-8d602288d901 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:25.358298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c32cd0-c643-4622-a771-8d602288d901 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:25.384348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:25.385253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:25.385621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:25.385959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] PATH_INFO: `/v3/projects/bc0db5d97aee425dbfc522a22162a6b1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:25.388094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc0db5d97aee425dbfc522a22162a6b1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:25.390108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:25.463612 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd2898c1e2740b3985343d7daf1361c', 'request_id': 'req-a416d3f4-47e9-48b6-8898-b7448174495d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.483844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd2898c1e2740b3985343d7daf1361c', 'request_id': 'req-a416d3f4-47e9-48b6-8898-b7448174495d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.500930 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd2898c1e2740b3985343d7daf1361c', 'request_id': 'req-a416d3f4-47e9-48b6-8898-b7448174495d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.525712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc0db5d97aee425dbfc522a22162a6b1', 'request_id': 'req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.537442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] The token cache is being invalidate because project dfd2898c1e2740b3985343d7daf1361c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:25.537869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.539818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.542383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 827/1665] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:25 2022] DELETE /identity/v3/projects/dfd2898c1e2740b3985343d7daf1361c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:25.543256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:25.548178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:25.551133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc0db5d97aee425dbfc522a22162a6b1', 'request_id': 'req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.560421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc0db5d97aee425dbfc522a22162a6b1', 'request_id': 'req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.593346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] The token cache is being invalidate because project bc0db5d97aee425dbfc522a22162a6b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:25.593996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.597041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cfa69bbb-6156-4b1a-a88e-6f8dd7bdb795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.601259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 839/1666] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:25 2022] DELETE /identity/v3/projects/bc0db5d97aee425dbfc522a22162a6b1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:25.604940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3067102-3079-40c8-a5ed-cddaa2e9a926 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:25.606254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3067102-3079-40c8-a5ed-cddaa2e9a926 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:25.606254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3067102-3079-40c8-a5ed-cddaa2e9a926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:25.643916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:25.650010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e3067102-3079-40c8-a5ed-cddaa2e9a926 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:25.655395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e3067102-3079-40c8-a5ed-cddaa2e9a926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:25.686311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416d3f4-47e9-48b6-8898-b7448174495d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:25.781702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:25.783166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:25.783166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:25.783325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] PATH_INFO: `/v3/projects/6a8e35f84ffb489cbfd0d18d29451367` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:25.788621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8e35f84ffb489cbfd0d18d29451367)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:25.791210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:25.838672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 840/1667] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:25.843412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c30fb-91d1-4af7-b456-78fce1695a17 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:25.844022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c30fb-91d1-4af7-b456-78fce1695a17 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:25.844516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c30fb-91d1-4af7-b456-78fce1695a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:25.883623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-418c30fb-91d1-4af7-b456-78fce1695a17 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:25.885505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-418c30fb-91d1-4af7-b456-78fce1695a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:25.929592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8e35f84ffb489cbfd0d18d29451367', 'request_id': 'req-a725a135-067e-42fd-bd50-9196b342614f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.963317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8e35f84ffb489cbfd0d18d29451367', 'request_id': 'req-a725a135-067e-42fd-bd50-9196b342614f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.978203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8e35f84ffb489cbfd0d18d29451367', 'request_id': 'req-a725a135-067e-42fd-bd50-9196b342614f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:25.992953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:26.005666 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] The token cache is being invalidate because project 6a8e35f84ffb489cbfd0d18d29451367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:26.005988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:26.007301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a725a135-067e-42fd-bd50-9196b342614f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:26.018802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 828/1668] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:25 2022] DELETE /identity/v3/projects/6a8e35f84ffb489cbfd0d18d29451367 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:26.027043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32c0a7-0ce3-420a-b121-f9ffc6a2c634 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.027993 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32c0a7-0ce3-420a-b121-f9ffc6a2c634 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.028484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32c0a7-0ce3-420a-b121-f9ffc6a2c634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.099234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9c32c0a7-0ce3-420a-b121-f9ffc6a2c634 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:26.101843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9c32c0a7-0ce3-420a-b121-f9ffc6a2c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:26.109298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 841/1669] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:26.117227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6997b-e4a7-4740-8b6c-650468ef5679 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.123671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6997b-e4a7-4740-8b6c-650468ef5679 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.124082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6997b-e4a7-4740-8b6c-650468ef5679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.130792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 829/1670] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:26.132234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f401-a39c-4810-883d-a2aea774c4f1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.133705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f401-a39c-4810-883d-a2aea774c4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.133878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f401-a39c-4810-883d-a2aea774c4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.212749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b7c9f401-a39c-4810-883d-a2aea774c4f1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:26.214610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b7c9f401-a39c-4810-883d-a2aea774c4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:26.235209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 830/1671] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:26.236496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9f401-a39c-4810-883d-a2aea774c4f1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:26.258840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-fcc6997b-e4a7-4740-8b6c-650468ef5679 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:26.269692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fcc6997b-e4a7-4740-8b6c-650468ef5679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:26.275997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9f401-a39c-4810-883d-a2aea774c4f1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:26.286203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca299b38-0b11-40b2-ad64-35b3d8ab23b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:26.287186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca299b38-0b11-40b2-ad64-35b3d8ab23b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.289327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca299b38-0b11-40b2-ad64-35b3d8ab23b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.289783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca299b38-0b11-40b2-ad64-35b3d8ab23b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.291149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca299b38-0b11-40b2-ad64-35b3d8ab23b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:26.293100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca299b38-0b11-40b2-ad64-35b3d8ab23b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:26.298250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca299b38-0b11-40b2-ad64-35b3d8ab23b1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:26.307978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 842/1672] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:26.309176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6997b-e4a7-4740-8b6c-650468ef5679 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:26.329565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 831/1673] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:26.335385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63778f6-d036-4732-b8fb-14ebb9fe1d83 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.336349 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6997b-e4a7-4740-8b6c-650468ef5679 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:26.340158 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63778f6-d036-4732-b8fb-14ebb9fe1d83 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.340689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63778f6-d036-4732-b8fb-14ebb9fe1d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.355258 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b0236-9173-454e-8f3b-908eccb9aed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:26.356063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b0236-9173-454e-8f3b-908eccb9aed9 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.356418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b0236-9173-454e-8f3b-908eccb9aed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.356767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b0236-9173-454e-8f3b-908eccb9aed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.357588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b0236-9173-454e-8f3b-908eccb9aed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:26.357966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b0236-9173-454e-8f3b-908eccb9aed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:26.362748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b0236-9173-454e-8f3b-908eccb9aed9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:26.391691 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 843/1674] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:26.396609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b0236-9173-454e-8f3b-908eccb9aed9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:26.400423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f63778f6-d036-4732-b8fb-14ebb9fe1d83 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:26.401975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f63778f6-d036-4732-b8fb-14ebb9fe1d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:26.421417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b0236-9173-454e-8f3b-908eccb9aed9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:26.428316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 832/1675] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:26.429526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63778f6-d036-4732-b8fb-14ebb9fe1d83 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:26.436791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483fbdf-9876-4955-a725-58df4ba91810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:26.437761 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483fbdf-9876-4955-a725-58df4ba91810 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.438165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483fbdf-9876-4955-a725-58df4ba91810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.438517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483fbdf-9876-4955-a725-58df4ba91810 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.439296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483fbdf-9876-4955-a725-58df4ba91810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:26.441350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483fbdf-9876-4955-a725-58df4ba91810 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:26.470378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63778f6-d036-4732-b8fb-14ebb9fe1d83 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:26.485514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390ba42-536c-405a-8c92-05f4979adaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:26.486443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390ba42-536c-405a-8c92-05f4979adaec admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.486850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390ba42-536c-405a-8c92-05f4979adaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.487383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390ba42-536c-405a-8c92-05f4979adaec admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.489896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390ba42-536c-405a-8c92-05f4979adaec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:26.493046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390ba42-536c-405a-8c92-05f4979adaec admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:26.524964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 844/1676] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:26 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:26.529751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483fbdf-9876-4955-a725-58df4ba91810 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:26.564890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 833/1677] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:26 2022] POST /identity/v3/projects => generated 403 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:26.564890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390ba42-536c-405a-8c92-05f4979adaec admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:26.647983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:26.708436 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483fbdf-9876-4955-a725-58df4ba91810 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:26.738983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390ba42-536c-405a-8c92-05f4979adaec admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:26.750021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c42ac-3c2b-49a1-b9f4-e319cf23b762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:26.751881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c42ac-3c2b-49a1-b9f4-e319cf23b762 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.752174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c42ac-3c2b-49a1-b9f4-e319cf23b762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.752465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c42ac-3c2b-49a1-b9f4-e319cf23b762 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.753175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c42ac-3c2b-49a1-b9f4-e319cf23b762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:26.756467 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c42ac-3c2b-49a1-b9f4-e319cf23b762 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:26.804078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 834/1678] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:26 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:26.804675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c42ac-3c2b-49a1-b9f4-e319cf23b762 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:26.830307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c42ac-3c2b-49a1-b9f4-e319cf23b762 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:26.841585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254b319-d588-4f79-a5e0-7268428c6ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:26.842430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254b319-d588-4f79-a5e0-7268428c6ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.842822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254b319-d588-4f79-a5e0-7268428c6ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.843215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254b319-d588-4f79-a5e0-7268428c6ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.843906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254b319-d588-4f79-a5e0-7268428c6ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:26.845931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254b319-d588-4f79-a5e0-7268428c6ea2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:26.859857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fec16-b239-48a8-aa96-1e70ce603a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:26.860561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fec16-b239-48a8-aa96-1e70ce603a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.861003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fec16-b239-48a8-aa96-1e70ce603a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.861288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fec16-b239-48a8-aa96-1e70ce603a26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.862039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fec16-b239-48a8-aa96-1e70ce603a26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:26.862378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fec16-b239-48a8-aa96-1e70ce603a26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:26.864015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fec16-b239-48a8-aa96-1e70ce603a26 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:26.894748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 835/1679] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:26 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:26.898731 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 845/1680] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:26.898911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254b319-d588-4f79-a5e0-7268428c6ea2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:26.905864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fec16-b239-48a8-aa96-1e70ce603a26 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:26.932508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254b319-d588-4f79-a5e0-7268428c6ea2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:26.935074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fec16-b239-48a8-aa96-1e70ce603a26 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:26.944035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d9242-bba5-4134-b350-9321a707fb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:26.944782 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d9242-bba5-4134-b350-9321a707fb63 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.945084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d9242-bba5-4134-b350-9321a707fb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.945934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d9242-bba5-4134-b350-9321a707fb63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.946965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d9242-bba5-4134-b350-9321a707fb63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:26.948433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d9242-bba5-4134-b350-9321a707fb63 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:26.954405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e181c03-3d30-45bd-a27d-7500daf350db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:26.955165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e181c03-3d30-45bd-a27d-7500daf350db admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:26.955448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e181c03-3d30-45bd-a27d-7500daf350db admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:26.955727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e181c03-3d30-45bd-a27d-7500daf350db admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:26.956471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e181c03-3d30-45bd-a27d-7500daf350db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:26.964382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e181c03-3d30-45bd-a27d-7500daf350db admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:26.985631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 836/1681] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:26.990738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d9242-bba5-4134-b350-9321a707fb63 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:27.007770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 846/1682] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:27.008705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e181c03-3d30-45bd-a27d-7500daf350db admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:27.017272 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d9242-bba5-4134-b350-9321a707fb63 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:27.025595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45a0a7-647c-4b37-a14f-642fbbe3b77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:27.026841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45a0a7-647c-4b37-a14f-642fbbe3b77a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:27.027198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45a0a7-647c-4b37-a14f-642fbbe3b77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:27.027540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45a0a7-647c-4b37-a14f-642fbbe3b77a admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:27.028320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45a0a7-647c-4b37-a14f-642fbbe3b77a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:27.032585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e181c03-3d30-45bd-a27d-7500daf350db admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:27.033815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45a0a7-647c-4b37-a14f-642fbbe3b77a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:27.046785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9e119-f03a-4829-89f6-421abb579777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:27.048502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9e119-f03a-4829-89f6-421abb579777 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:27.048806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9e119-f03a-4829-89f6-421abb579777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:27.049065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9e119-f03a-4829-89f6-421abb579777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:27.049831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9e119-f03a-4829-89f6-421abb579777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:27.070290 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9e119-f03a-4829-89f6-421abb579777 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:27.086991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 837/1683] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:26 2022] POST /identity/v3/projects => generated 355 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:27.087990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45a0a7-647c-4b37-a14f-642fbbe3b77a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:27.113837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 847/1684] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:27.115289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9e119-f03a-4829-89f6-421abb579777 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:27.164073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9e119-f03a-4829-89f6-421abb579777 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:27.205883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45a0a7-647c-4b37-a14f-642fbbe3b77a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:27.212720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc092d-aa21-4799-b5b1-95da1cb8a33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:27.213814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:27.214906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc092d-aa21-4799-b5b1-95da1cb8a33c admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:27.216420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc092d-aa21-4799-b5b1-95da1cb8a33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:27.217045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc092d-aa21-4799-b5b1-95da1cb8a33c admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:27.217974 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc092d-aa21-4799-b5b1-95da1cb8a33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:27.222068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc092d-aa21-4799-b5b1-95da1cb8a33c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:27.241044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 838/1685] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:27.242256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc092d-aa21-4799-b5b1-95da1cb8a33c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:27.264977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc092d-aa21-4799-b5b1-95da1cb8a33c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:27.277530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4197199-7b39-485a-83b1-6661875073b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:27.278492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4197199-7b39-485a-83b1-6661875073b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:27.278769 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4197199-7b39-485a-83b1-6661875073b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:27.279040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4197199-7b39-485a-83b1-6661875073b1 admin admin] PATH_INFO: `/v3/projects/c371c880db814276ac724b8d620c878e/users/371c1d2225004daba2922154030cf3c1/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:27.323435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4197199-7b39-485a-83b1-6661875073b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c371c880db814276ac724b8d620c878e, user_id=371c1d2225004daba2922154030cf3c1, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:27.326923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4197199-7b39-485a-83b1-6661875073b1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:27.352185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495ea95-0929-460b-8f3e-40b4a8b97bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:27.352982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495ea95-0929-460b-8f3e-40b4a8b97bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:27.353271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495ea95-0929-460b-8f3e-40b4a8b97bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:27.353614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495ea95-0929-460b-8f3e-40b4a8b97bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:27.354371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495ea95-0929-460b-8f3e-40b4a8b97bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:27.358125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495ea95-0929-460b-8f3e-40b4a8b97bb8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:27.369798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 839/1686] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:27 2022] PUT /identity/v3/projects/c371c880db814276ac724b8d620c878e/users/371c1d2225004daba2922154030cf3c1/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:27.376308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4197199-7b39-485a-83b1-6661875073b1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:27.404994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 848/1687] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:27 2022] POST /identity/v3/users => generated 278 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:27.407099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b0bc1-0894-435d-a774-2ba77ca79650 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:27.408551 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b0bc1-0894-435d-a774-2ba77ca79650 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:27.408833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b0bc1-0894-435d-a774-2ba77ca79650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:27.472831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-630b0bc1-0894-435d-a774-2ba77ca79650 None None] MFA Rules not processed for user `371c1d2225004daba2922154030cf3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:27.473877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-630b0bc1-0894-435d-a774-2ba77ca79650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:27.504101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:27.557661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4197199-7b39-485a-83b1-6661875073b1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:27.571448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a49d26-234d-4a97-aea8-d5e3bc56f2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:27.572557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a49d26-234d-4a97-aea8-d5e3bc56f2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:27.572948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a49d26-234d-4a97-aea8-d5e3bc56f2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:27.573383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a49d26-234d-4a97-aea8-d5e3bc56f2dd admin admin] PATH_INFO: `/v3/projects/cb0232fa025246b9a0f4fc1a8a6e328a/users/b9ced524933d4bcf9da27c0a2aaf3175/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:27.611367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a49d26-234d-4a97-aea8-d5e3bc56f2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0232fa025246b9a0f4fc1a8a6e328a, user_id=b9ced524933d4bcf9da27c0a2aaf3175, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:27.619492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a49d26-234d-4a97-aea8-d5e3bc56f2dd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:27.649688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 849/1688] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:27.649688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b0bc1-0894-435d-a774-2ba77ca79650 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:27.672122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 840/1689] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:27 2022] PUT /identity/v3/projects/cb0232fa025246b9a0f4fc1a8a6e328a/users/b9ced524933d4bcf9da27c0a2aaf3175/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:27.672976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a49d26-234d-4a97-aea8-d5e3bc56f2dd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:27.683420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b0bc1-0894-435d-a774-2ba77ca79650 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:27.821072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a49d26-234d-4a97-aea8-d5e3bc56f2dd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:27.834017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b16e-2211-4d88-b1ae-a85aa55561bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:27.834981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b16e-2211-4d88-b1ae-a85aa55561bd admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:27.835357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b16e-2211-4d88-b1ae-a85aa55561bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:27.835729 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b16e-2211-4d88-b1ae-a85aa55561bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:27.839459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f0a9-c30f-405b-89b0-395f58288f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:27.841086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f0a9-c30f-405b-89b0-395f58288f5f admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:27.842563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421b16e-2211-4d88-b1ae-a85aa55561bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:27.844315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f0a9-c30f-405b-89b0-395f58288f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:27.845123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421b16e-2211-4d88-b1ae-a85aa55561bd admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:27.846350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f0a9-c30f-405b-89b0-395f58288f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:27.846798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246f0a9-c30f-405b-89b0-395f58288f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:27.848674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246f0a9-c30f-405b-89b0-395f58288f5f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:27.860109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 850/1690] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:27.862184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f874795-98ea-41c3-b67c-8e4090427345 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:27.862824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f874795-98ea-41c3-b67c-8e4090427345 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:27.863311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f874795-98ea-41c3-b67c-8e4090427345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:27.898537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4f874795-98ea-41c3-b67c-8e4090427345 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:27.900044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4f874795-98ea-41c3-b67c-8e4090427345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:27.925352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 851/1691] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:27.926480 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f874795-98ea-41c3-b67c-8e4090427345 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:27.943438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 841/1692] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:27 2022] POST /identity/v3/projects => generated 381 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:27.945180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f0a9-c30f-405b-89b0-395f58288f5f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:27.952846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f874795-98ea-41c3-b67c-8e4090427345 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:28.007988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:28.062284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f0a9-c30f-405b-89b0-395f58288f5f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:28.068005 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39f6a8-00c2-4cde-bb43-461e538aca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:28.068911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39f6a8-00c2-4cde-bb43-461e538aca31 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:28.069196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39f6a8-00c2-4cde-bb43-461e538aca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:28.069473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39f6a8-00c2-4cde-bb43-461e538aca31 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:28.070354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39f6a8-00c2-4cde-bb43-461e538aca31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:28.072756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39f6a8-00c2-4cde-bb43-461e538aca31 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:28.097408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6f19f-5e05-4ca0-bd2b-491c08127e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:28.098280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6f19f-5e05-4ca0-bd2b-491c08127e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:28.098560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6f19f-5e05-4ca0-bd2b-491c08127e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:28.098834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6f19f-5e05-4ca0-bd2b-491c08127e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:28.102751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6f19f-5e05-4ca0-bd2b-491c08127e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:28.105772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6f19f-5e05-4ca0-bd2b-491c08127e5c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:28.106842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 842/1693] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:27 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:28.107870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39f6a8-00c2-4cde-bb43-461e538aca31 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:28.110693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0c39f6a8-00c2-4cde-bb43-461e538aca31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:28.133519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 852/1694] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:28.135472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464e31d-0bde-4f52-8db6-9d1fd540193b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:28.135472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464e31d-0bde-4f52-8db6-9d1fd540193b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:28.135790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464e31d-0bde-4f52-8db6-9d1fd540193b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:28.191129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:28.201604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6464e31d-0bde-4f52-8db6-9d1fd540193b None None] MFA Rules not processed for user `14d3e513d8ce4efc87a359e04a77f05b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:28.202648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6464e31d-0bde-4f52-8db6-9d1fd540193b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:28.239785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39f6a8-00c2-4cde-bb43-461e538aca31 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:28.249568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79316b73-4d34-4c65-9164-002fdba6cafc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:28.251573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79316b73-4d34-4c65-9164-002fdba6cafc service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:28.251883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79316b73-4d34-4c65-9164-002fdba6cafc service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:28.252167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79316b73-4d34-4c65-9164-002fdba6cafc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:28.277394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79316b73-4d34-4c65-9164-002fdba6cafc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:28.281129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79316b73-4d34-4c65-9164-002fdba6cafc service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:28.319494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 843/1695] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:28.321046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79316b73-4d34-4c65-9164-002fdba6cafc service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:28.353351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79316b73-4d34-4c65-9164-002fdba6cafc service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:28.365212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa50ae-a465-47ad-8c9a-92461a52de3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:28.366913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa50ae-a465-47ad-8c9a-92461a52de3b admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:28.367183 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa50ae-a465-47ad-8c9a-92461a52de3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:28.367998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa50ae-a465-47ad-8c9a-92461a52de3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:28.369999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa50ae-a465-47ad-8c9a-92461a52de3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:28.374521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa50ae-a465-47ad-8c9a-92461a52de3b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:28.387460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 853/1696] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:28.403888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464e31d-0bde-4f52-8db6-9d1fd540193b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:28.416801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 844/1697] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:28.420321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a146d26-5eb2-4e82-a8db-dc46671ee724 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:28.420575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a146d26-5eb2-4e82-a8db-dc46671ee724 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:28.420854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a146d26-5eb2-4e82-a8db-dc46671ee724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:28.427999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464e31d-0bde-4f52-8db6-9d1fd540193b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:28.442079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5e666-b08b-4252-8c94-bfe1b7cdcc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:28.444429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5e666-b08b-4252-8c94-bfe1b7cdcc90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:28.444429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5e666-b08b-4252-8c94-bfe1b7cdcc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:28.446294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5e666-b08b-4252-8c94-bfe1b7cdcc90 admin admin] PATH_INFO: `/v3/projects/032ec5272572486c98dd352dd9635638/users/c47aaccb33a64a9a99f06ad3d671c712/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:28.473903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7a146d26-5eb2-4e82-a8db-dc46671ee724 None None] MFA Rules not processed for user `14d3e513d8ce4efc87a359e04a77f05b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:28.483828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7a146d26-5eb2-4e82-a8db-dc46671ee724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:28.502455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5e666-b08b-4252-8c94-bfe1b7cdcc90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032ec5272572486c98dd352dd9635638, user_id=c47aaccb33a64a9a99f06ad3d671c712, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:28.508865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 845/1698] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:28.516417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a146d26-5eb2-4e82-a8db-dc46671ee724 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:28.517984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5e666-b08b-4252-8c94-bfe1b7cdcc90 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:28.546960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 854/1699] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:28 2022] PUT /identity/v3/projects/032ec5272572486c98dd352dd9635638/users/c47aaccb33a64a9a99f06ad3d671c712/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:28.556241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5e666-b08b-4252-8c94-bfe1b7cdcc90 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:28.556241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e8c5e666-b08b-4252-8c94-bfe1b7cdcc90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:28.560049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a146d26-5eb2-4e82-a8db-dc46671ee724 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:28.729420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b89987-8f03-4ebb-a5a1-aede3dd97625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:28.731891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b89987-8f03-4ebb-a5a1-aede3dd97625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:28.732690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b89987-8f03-4ebb-a5a1-aede3dd97625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:28.734916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b89987-8f03-4ebb-a5a1-aede3dd97625 admin admin] PATH_INFO: `/v3/projects/a0f2168568bb424d87f528585f4a576a/users/bcc5cc4402d9406eabaed6804ad33c3d/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:28.765407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5e666-b08b-4252-8c94-bfe1b7cdcc90 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:28.772342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b89987-8f03-4ebb-a5a1-aede3dd97625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f2168568bb424d87f528585f4a576a, user_id=bcc5cc4402d9406eabaed6804ad33c3d, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:28.779867 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05806c6-365a-4ddf-bafd-30a3205eaca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:28.780011 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b89987-8f03-4ebb-a5a1-aede3dd97625 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:28.780348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05806c6-365a-4ddf-bafd-30a3205eaca0 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:28.782160 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05806c6-365a-4ddf-bafd-30a3205eaca0 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:28.782701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05806c6-365a-4ddf-bafd-30a3205eaca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:28.824742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 846/1700] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:28 2022] PUT /identity/v3/projects/a0f2168568bb424d87f528585f4a576a/users/bcc5cc4402d9406eabaed6804ad33c3d/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:28.828236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0655418-7a48-4b4c-8d29-8f19f5b17a89 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:28.829489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0655418-7a48-4b4c-8d29-8f19f5b17a89 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:28.832492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0655418-7a48-4b4c-8d29-8f19f5b17a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:28.882199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:28.950806 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a0655418-7a48-4b4c-8d29-8f19f5b17a89 None None] MFA Rules not processed for user `c47aaccb33a64a9a99f06ad3d671c712`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:28.955222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a0655418-7a48-4b4c-8d29-8f19f5b17a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:28.997841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05806c6-365a-4ddf-bafd-30a3205eaca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:28.999892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05806c6-365a-4ddf-bafd-30a3205eaca0 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:29.040404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 855/1701] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:29.042433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05806c6-365a-4ddf-bafd-30a3205eaca0 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:29.083365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05806c6-365a-4ddf-bafd-30a3205eaca0 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:29.151126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 847/1702] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:29.155993 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0655418-7a48-4b4c-8d29-8f19f5b17a89 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:29.238499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99be8c-ebc3-472c-b828-51953624de74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:29.239228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99be8c-ebc3-472c-b828-51953624de74 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:29.239513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99be8c-ebc3-472c-b828-51953624de74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:29.239791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99be8c-ebc3-472c-b828-51953624de74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:29.241041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99be8c-ebc3-472c-b828-51953624de74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:29.241588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:29.249539 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99be8c-ebc3-472c-b828-51953624de74 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:29.275758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 856/1703] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:29.279842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99be8c-ebc3-472c-b828-51953624de74 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:29.303108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99be8c-ebc3-472c-b828-51953624de74 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:29.318490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73865a52-f95e-4ee9-85fa-6841150072f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:29.319427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865a52-f95e-4ee9-85fa-6841150072f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:29.319709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865a52-f95e-4ee9-85fa-6841150072f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:29.319985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865a52-f95e-4ee9-85fa-6841150072f2 admin admin] PATH_INFO: `/v3/projects/a0f2168568bb424d87f528585f4a576a/users/bcc5cc4402d9406eabaed6804ad33c3d/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:29.327658 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73865a52-f95e-4ee9-85fa-6841150072f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f2168568bb424d87f528585f4a576a, user_id=bcc5cc4402d9406eabaed6804ad33c3d, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:29.330377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73865a52-f95e-4ee9-85fa-6841150072f2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:29.330500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0655418-7a48-4b4c-8d29-8f19f5b17a89 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:29.344213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b86f7-f689-4872-a67d-00c8160911d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:29.344924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b86f7-f689-4872-a67d-00c8160911d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:29.345205 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b86f7-f689-4872-a67d-00c8160911d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:29.345479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b86f7-f689-4872-a67d-00c8160911d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:29.346233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b86f7-f689-4872-a67d-00c8160911d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:29.348194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b86f7-f689-4872-a67d-00c8160911d1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:29.367054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 857/1704] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:29 2022] PUT /identity/v3/projects/a0f2168568bb424d87f528585f4a576a/users/bcc5cc4402d9406eabaed6804ad33c3d/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:29.378669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b574a-2dc4-4b49-a739-8fe4ec1ff108 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:29.378964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b574a-2dc4-4b49-a739-8fe4ec1ff108 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:29.379242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b574a-2dc4-4b49-a739-8fe4ec1ff108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:29.420832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 848/1705] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:29 2022] POST /identity/v3/projects => generated 355 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:29.424454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b86f7-f689-4872-a67d-00c8160911d1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:29.451577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-482b574a-2dc4-4b49-a739-8fe4ec1ff108 None None] MFA Rules not processed for user `bcc5cc4402d9406eabaed6804ad33c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:29.452818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-482b574a-2dc4-4b49-a739-8fe4ec1ff108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:29.584425 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b86f7-f689-4872-a67d-00c8160911d1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:29.597401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 858/1706] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:04:29.598840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bce59-505d-45f1-a456-59833b23e0ed None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:29.599177 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bce59-505d-45f1-a456-59833b23e0ed None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:29.599403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bce59-505d-45f1-a456-59833b23e0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:29.610667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f93d64-de75-4e5e-8aac-6d42a8b33286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:29.611375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f93d64-de75-4e5e-8aac-6d42a8b33286 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:29.611654 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f93d64-de75-4e5e-8aac-6d42a8b33286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:29.611930 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f93d64-de75-4e5e-8aac-6d42a8b33286 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:29.612633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f93d64-de75-4e5e-8aac-6d42a8b33286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:29.627677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f93d64-de75-4e5e-8aac-6d42a8b33286 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:29.677504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-101bce59-505d-45f1-a456-59833b23e0ed None None] MFA Rules not processed for user `48e77af2b3a2409fa6d352b0d2674b85`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:29.680251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-101bce59-505d-45f1-a456-59833b23e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:29.685773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 849/1707] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:29 2022] POST /identity/v3/users => generated 284 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:29.685773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f93d64-de75-4e5e-8aac-6d42a8b33286 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:29.692402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e9f93d64-de75-4e5e-8aac-6d42a8b33286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:29.755175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:29.910239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f93d64-de75-4e5e-8aac-6d42a8b33286 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:29.911493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 859/1708] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:29.912567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bce59-505d-45f1-a456-59833b23e0ed None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:29.922823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bf29a-b104-49f7-a240-f5e9e9026391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:29.927130 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf29a-b104-49f7-a240-f5e9e9026391 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:29.927527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf29a-b104-49f7-a240-f5e9e9026391 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:29.927928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf29a-b104-49f7-a240-f5e9e9026391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:29.947826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bce59-505d-45f1-a456-59833b23e0ed None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:29.954701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bf29a-b104-49f7-a240-f5e9e9026391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:29.956806 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bf29a-b104-49f7-a240-f5e9e9026391 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:29.966049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff084efb-4b24-4036-a379-65ed4c4b173e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:29.966931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff084efb-4b24-4036-a379-65ed4c4b173e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:29.967322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff084efb-4b24-4036-a379-65ed4c4b173e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:29.968052 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff084efb-4b24-4036-a379-65ed4c4b173e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:29.968914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff084efb-4b24-4036-a379-65ed4c4b173e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:29.972908 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff084efb-4b24-4036-a379-65ed4c4b173e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:29.991179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 850/1709] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:29.993615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bf29a-b104-49f7-a240-f5e9e9026391 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:29.999187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 860/1710] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:30.004975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bb4bf29a-b104-49f7-a240-f5e9e9026391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:30.007203 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff084efb-4b24-4036-a379-65ed4c4b173e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:30.043545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff084efb-4b24-4036-a379-65ed4c4b173e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:30.064042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a04da9-9c2c-4e34-8c9c-b3c495a2fe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:30.065078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a04da9-9c2c-4e34-8c9c-b3c495a2fe34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:30.066319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a04da9-9c2c-4e34-8c9c-b3c495a2fe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:30.067314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a04da9-9c2c-4e34-8c9c-b3c495a2fe34 admin admin] PATH_INFO: `/v3/projects/2d38a45c88e748dd92091220957cc75d/users/0f51ac1db66f48ec97770d8e7ad2737a/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:30.121157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a04da9-9c2c-4e34-8c9c-b3c495a2fe34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d38a45c88e748dd92091220957cc75d, user_id=0f51ac1db66f48ec97770d8e7ad2737a, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:30.127299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a04da9-9c2c-4e34-8c9c-b3c495a2fe34 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:30.173091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:30.180651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 861/1711] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:30 2022] PUT /identity/v3/projects/2d38a45c88e748dd92091220957cc75d/users/0f51ac1db66f48ec97770d8e7ad2737a/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:30.224573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb050d-ad9d-4cf4-99b7-7f0aa7ca38a6 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:30.225004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb050d-ad9d-4cf4-99b7-7f0aa7ca38a6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:30.225287 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb050d-ad9d-4cf4-99b7-7f0aa7ca38a6 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:30.226852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 862/1712] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jun 20 11:04:30 2022] GET /identity => generated 270 bytes in 46 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:30.243628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb050d-ad9d-4cf4-99b7-7f0aa7ca38a6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:30.256543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bf29a-b104-49f7-a240-f5e9e9026391 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:30.379206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb050d-ad9d-4cf4-99b7-7f0aa7ca38a6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:30.384420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b01a33-0a2d-4fb8-98b6-1fec207e5445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:30.385359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b01a33-0a2d-4fb8-98b6-1fec207e5445 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:30.385933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b01a33-0a2d-4fb8-98b6-1fec207e5445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:30.386568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b01a33-0a2d-4fb8-98b6-1fec207e5445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:30.387598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b01a33-0a2d-4fb8-98b6-1fec207e5445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:30.391756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b01a33-0a2d-4fb8-98b6-1fec207e5445 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:30.410866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 863/1713] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:30.418968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e833feb-8e9b-4589-a9c2-781c1eaedc41 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:30.419261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e833feb-8e9b-4589-a9c2-781c1eaedc41 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:30.419570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e833feb-8e9b-4589-a9c2-781c1eaedc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:30.500245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39b18d-e34a-4dfe-ac16-c60cf0ff4424 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:30.500963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b18d-e34a-4dfe-ac16-c60cf0ff4424 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:30.501245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b18d-e34a-4dfe-ac16-c60cf0ff4424 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:30.501540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39b18d-e34a-4dfe-ac16-c60cf0ff4424 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:30.543553 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6e833feb-8e9b-4589-a9c2-781c1eaedc41 None None] MFA Rules not processed for user `d73fe529c7d342f895aabcb9a39aba78`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:30.545217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6e833feb-8e9b-4589-a9c2-781c1eaedc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:30.681864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39b18d-e34a-4dfe-ac16-c60cf0ff4424 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:30.692624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39b18d-e34a-4dfe-ac16-c60cf0ff4424 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:30.729620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 864/1714] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:30.731554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e833feb-8e9b-4589-a9c2-781c1eaedc41 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:30.751771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 851/1715] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 757 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:30.752939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39b18d-e34a-4dfe-ac16-c60cf0ff4424 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:30.786230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:30.791868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39b18d-e34a-4dfe-ac16-c60cf0ff4424 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:30.808808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0f969-e515-4b11-84fa-66ae67d89bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:30.809759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0f969-e515-4b11-84fa-66ae67d89bec admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:30.810041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0f969-e515-4b11-84fa-66ae67d89bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:30.810369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0f969-e515-4b11-84fa-66ae67d89bec admin admin] PATH_INFO: `/v3/projects/2d38a45c88e748dd92091220957cc75d/users/0f51ac1db66f48ec97770d8e7ad2737a/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:30.820320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0f969-e515-4b11-84fa-66ae67d89bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d38a45c88e748dd92091220957cc75d, user_id=0f51ac1db66f48ec97770d8e7ad2737a, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:30.822581 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0f969-e515-4b11-84fa-66ae67d89bec admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:30.859584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 852/1716] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:30 2022] PUT /identity/v3/projects/2d38a45c88e748dd92091220957cc75d/users/0f51ac1db66f48ec97770d8e7ad2737a/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:30.861232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a6064-3818-4548-b0c3-5705f2ecccc3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:30.861511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a6064-3818-4548-b0c3-5705f2ecccc3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:30.861790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a6064-3818-4548-b0c3-5705f2ecccc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:30.934894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9c4a6064-3818-4548-b0c3-5705f2ecccc3 None None] MFA Rules not processed for user `b9ced524933d4bcf9da27c0a2aaf3175`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:30.938370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9c4a6064-3818-4548-b0c3-5705f2ecccc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:31.137371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:31.151685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 853/1717] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:31.154158 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a6064-3818-4548-b0c3-5705f2ecccc3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:31.281726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e833feb-8e9b-4589-a9c2-781c1eaedc41 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:31.285212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a6064-3818-4548-b0c3-5705f2ecccc3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:31.295059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b324f-d063-496c-bea6-64803e898018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:31.295754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b324f-d063-496c-bea6-64803e898018 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:31.297305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b324f-d063-496c-bea6-64803e898018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:31.297686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b324f-d063-496c-bea6-64803e898018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:31.314404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b324f-d063-496c-bea6-64803e898018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:31.315903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b324f-d063-496c-bea6-64803e898018 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:31.340040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 854/1718] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:31.341290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64013a5e-9525-46d3-8b3e-66574ff50835 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:31.341696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64013a5e-9525-46d3-8b3e-66574ff50835 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:31.342797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64013a5e-9525-46d3-8b3e-66574ff50835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:31.443520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bc6c7-e168-468c-8d86-ccd918442010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:31.444218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc6c7-e168-468c-8d86-ccd918442010 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:31.444488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc6c7-e168-468c-8d86-ccd918442010 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:31.444826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc6c7-e168-468c-8d86-ccd918442010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:31.447996 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-64013a5e-9525-46d3-8b3e-66574ff50835 None None] MFA Rules not processed for user `0f51ac1db66f48ec97770d8e7ad2737a`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:31.449482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-64013a5e-9525-46d3-8b3e-66574ff50835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:31.607660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bc6c7-e168-468c-8d86-ccd918442010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:31.612843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bc6c7-e168-468c-8d86-ccd918442010 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:31.627195 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 855/1719] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:31.633395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64013a5e-9525-46d3-8b3e-66574ff50835 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:31.637329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 865/1720] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 907 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:31.645212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8293bb-de66-43c2-94d4-c69682fadca0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:31.645500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8293bb-de66-43c2-94d4-c69682fadca0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:31.645778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8293bb-de66-43c2-94d4-c69682fadca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:31.731864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2f8293bb-de66-43c2-94d4-c69682fadca0 None None] MFA Rules not processed for user `0c4a4ce86beb47c2b7910264a9238c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:31.747540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2f8293bb-de66-43c2-94d4-c69682fadca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:31.850303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64013a5e-9525-46d3-8b3e-66574ff50835 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:31.861652 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2399bd-c0cc-400a-bb93-2c7c5a5b0936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:31.863706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2399bd-c0cc-400a-bb93-2c7c5a5b0936 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:31.864000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2399bd-c0cc-400a-bb93-2c7c5a5b0936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:31.866323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2399bd-c0cc-400a-bb93-2c7c5a5b0936 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:31.868825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2399bd-c0cc-400a-bb93-2c7c5a5b0936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:31.875531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2399bd-c0cc-400a-bb93-2c7c5a5b0936 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:31.949281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 856/1721] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:31 2022] POST /identity/v3/projects => generated 403 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:31.953328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2399bd-c0cc-400a-bb93-2c7c5a5b0936 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:31.974156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 866/1722] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:31.987666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e1c92-5954-4171-8a85-7892f568cbd3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:31.987940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e1c92-5954-4171-8a85-7892f568cbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:31.988451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e1c92-5954-4171-8a85-7892f568cbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:31.998032 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:32.028764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ab2e1c92-5954-4171-8a85-7892f568cbd3 None None] MFA Rules not processed for user `0c4a4ce86beb47c2b7910264a9238c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:32.030043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ab2e1c92-5954-4171-8a85-7892f568cbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:32.067699 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:32.163227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2399bd-c0cc-400a-bb93-2c7c5a5b0936 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:32.189681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56493cf-cc0c-417e-bfdb-cd6352a522c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:32.191655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56493cf-cc0c-417e-bfdb-cd6352a522c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:32.191947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56493cf-cc0c-417e-bfdb-cd6352a522c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:32.192297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56493cf-cc0c-417e-bfdb-cd6352a522c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:32.197470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56493cf-cc0c-417e-bfdb-cd6352a522c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:32.207437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56493cf-cc0c-417e-bfdb-cd6352a522c6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:32.224356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 867/1723] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:04:32.234047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e1c92-5954-4171-8a85-7892f568cbd3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:32.239156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ab2e1c92-5954-4171-8a85-7892f568cbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:32.276947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 857/1724] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:31 2022] POST /identity/v3/users => generated 308 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:04:32.282387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56493cf-cc0c-417e-bfdb-cd6352a522c6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:32.309781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56493cf-cc0c-417e-bfdb-cd6352a522c6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:32.316688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9c643-1212-4d55-8fbd-9c8592cd4a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:32.317862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9c643-1212-4d55-8fbd-9c8592cd4a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:32.318367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9c643-1212-4d55-8fbd-9c8592cd4a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:32.318797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9c643-1212-4d55-8fbd-9c8592cd4a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:32.319693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9c643-1212-4d55-8fbd-9c8592cd4a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:32.322168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9c643-1212-4d55-8fbd-9c8592cd4a09 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:32.336702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 858/1725] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:32.338046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9c643-1212-4d55-8fbd-9c8592cd4a09 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:32.516213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e1c92-5954-4171-8a85-7892f568cbd3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:32.521455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee771fad-1973-4bb6-a474-007ce0d45f6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:32.522339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee771fad-1973-4bb6-a474-007ce0d45f6a service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:32.523742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee771fad-1973-4bb6-a474-007ce0d45f6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:32.524018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee771fad-1973-4bb6-a474-007ce0d45f6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:32.541360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9c643-1212-4d55-8fbd-9c8592cd4a09 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:32.565794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec3861-98bf-4346-82ce-9c824c58b275 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:32.566525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec3861-98bf-4346-82ce-9c824c58b275 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:32.567046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec3861-98bf-4346-82ce-9c824c58b275 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:32.567904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec3861-98bf-4346-82ce-9c824c58b275 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:32.594992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec3861-98bf-4346-82ce-9c824c58b275 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:32.600716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec3861-98bf-4346-82ce-9c824c58b275 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:32.630515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 859/1726] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:32.631466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec3861-98bf-4346-82ce-9c824c58b275 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:32.649801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee771fad-1973-4bb6-a474-007ce0d45f6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:32.654203 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee771fad-1973-4bb6-a474-007ce0d45f6a service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:32.658126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec3861-98bf-4346-82ce-9c824c58b275 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:32.669817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c0c77-e83b-4656-b2ce-ff31f0054261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:32.671155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c0c77-e83b-4656-b2ce-ff31f0054261 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:32.671560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c0c77-e83b-4656-b2ce-ff31f0054261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:32.671854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c0c77-e83b-4656-b2ce-ff31f0054261 admin admin] PATH_INFO: `/v3/projects/213d71ec77444306925e6afc1cad40b6/users/e9f4d8190ae644ae88ac0cf252e850f6/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:32.680019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 868/1727] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:32 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:32.683213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230a644-2bc4-4f29-a89a-b32864c7370d None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:32.683631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230a644-2bc4-4f29-a89a-b32864c7370d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:32.684972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230a644-2bc4-4f29-a89a-b32864c7370d None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:32.690026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 869/1728] 10.222.0.33 () {54 vars in 960 bytes} [Mon Jun 20 11:04:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:32.696347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c29137-c4cb-4e81-b92c-e9c3b3029265 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:32.696985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c29137-c4cb-4e81-b92c-e9c3b3029265 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:32.697398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c29137-c4cb-4e81-b92c-e9c3b3029265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:32.725405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c0c77-e83b-4656-b2ce-ff31f0054261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=213d71ec77444306925e6afc1cad40b6, user_id=e9f4d8190ae644ae88ac0cf252e850f6, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:32.734454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c0c77-e83b-4656-b2ce-ff31f0054261 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:32.752642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f0c29137-c4cb-4e81-b92c-e9c3b3029265 None None] MFA Rules not processed for user `bff19e37fa2243fda914f7f4e662c8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:32.755515 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f0c29137-c4cb-4e81-b92c-e9c3b3029265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:32.797864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 860/1729] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:32 2022] PUT /identity/v3/projects/213d71ec77444306925e6afc1cad40b6/users/e9f4d8190ae644ae88ac0cf252e850f6/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:32.802034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c0c77-e83b-4656-b2ce-ff31f0054261 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:32.945779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 870/1730] 10.222.0.33 () {58 vars in 1057 bytes} [Mon Jun 20 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:32.955229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c29137-c4cb-4e81-b92c-e9c3b3029265 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:32.969063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:32.981630 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c29137-c4cb-4e81-b92c-e9c3b3029265 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:32.995639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:32.998774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c0c77-e83b-4656-b2ce-ff31f0054261 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:33.011655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225876-149a-4b58-b2f3-4fe33558320b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:33.013246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225876-149a-4b58-b2f3-4fe33558320b admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:33.013586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225876-149a-4b58-b2f3-4fe33558320b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:33.013875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225876-149a-4b58-b2f3-4fe33558320b admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:33.016183 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c225876-149a-4b58-b2f3-4fe33558320b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:33.020132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c225876-149a-4b58-b2f3-4fe33558320b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:33.047470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 861/1731] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:33.051519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225876-149a-4b58-b2f3-4fe33558320b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:33.126414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa63b72-5bba-4c3b-946f-10f47cddab06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:33.127236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa63b72-5bba-4c3b-946f-10f47cddab06 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:33.127929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa63b72-5bba-4c3b-946f-10f47cddab06 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:33.129510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa63b72-5bba-4c3b-946f-10f47cddab06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:33.133661 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cfa63b72-5bba-4c3b-946f-10f47cddab06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:33.162693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa63b72-5bba-4c3b-946f-10f47cddab06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:33.170380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa63b72-5bba-4c3b-946f-10f47cddab06 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:33.189831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c225876-149a-4b58-b2f3-4fe33558320b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:33.199832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc4ed2-fbd1-47d7-a6c2-7aabf68ca962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:33.200219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 871/1732] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:33.201868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4ed2-fbd1-47d7-a6c2-7aabf68ca962 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:33.202379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa63b72-5bba-4c3b-946f-10f47cddab06 service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:33.203634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4ed2-fbd1-47d7-a6c2-7aabf68ca962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:33.204805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc4ed2-fbd1-47d7-a6c2-7aabf68ca962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:33.255700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa63b72-5bba-4c3b-946f-10f47cddab06 service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:33.273106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84338891-4f58-40ad-833e-ca80c1cf03cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:33.274144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84338891-4f58-40ad-833e-ca80c1cf03cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:33.275504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84338891-4f58-40ad-833e-ca80c1cf03cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:33.275722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84338891-4f58-40ad-833e-ca80c1cf03cf admin admin] PATH_INFO: `/v3/projects/213d71ec77444306925e6afc1cad40b6/users/e9f4d8190ae644ae88ac0cf252e850f6/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:33.283170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84338891-4f58-40ad-833e-ca80c1cf03cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=213d71ec77444306925e6afc1cad40b6, user_id=e9f4d8190ae644ae88ac0cf252e850f6, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:33.294071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84338891-4f58-40ad-833e-ca80c1cf03cf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:33.347959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 872/1733] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:33 2022] PUT /identity/v3/projects/213d71ec77444306925e6afc1cad40b6/users/e9f4d8190ae644ae88ac0cf252e850f6/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:33.413840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc4ed2-fbd1-47d7-a6c2-7aabf68ca962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:33.429960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc4ed2-fbd1-47d7-a6c2-7aabf68ca962 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:33.469015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 862/1734] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:33 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:04:34.972276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c9db7-3960-4e6d-a1b3-f6a12fedef64 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:34.972730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c9db7-3960-4e6d-a1b3-f6a12fedef64 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:34.972850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c9db7-3960-4e6d-a1b3-f6a12fedef64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:35.023342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-784c9db7-3960-4e6d-a1b3-f6a12fedef64 None None] MFA Rules not processed for user `0c4a4ce86beb47c2b7910264a9238c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:35.027406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-784c9db7-3960-4e6d-a1b3-f6a12fedef64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:35.149206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc4ed2-fbd1-47d7-a6c2-7aabf68ca962 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:35.152490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b5cc4ed2-fbd1-47d7-a6c2-7aabf68ca962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:35.192888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc4ed2-fbd1-47d7-a6c2-7aabf68ca962 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:35.293131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 873/1735] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:35.334979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:35.335970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:35.336347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:35.336711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] PATH_INFO: `/v3/users/6bd54a561aa948888095879de89cd98b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:35.338101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bd54a561aa948888095879de89cd98b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:35.341297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:35.405150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f453b0-ce10-47e3-b051-529fe9ac6415 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:35.405457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f453b0-ce10-47e3-b051-529fe9ac6415 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:35.405733 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f453b0-ce10-47e3-b051-529fe9ac6415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:35.423271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:35.428776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bd54a561aa948888095879de89cd98b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:35.429855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bd54a561aa948888095879de89cd98b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:35.466724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bd54a561aa948888095879de89cd98b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:35.467576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bd54a561aa948888095879de89cd98b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:35.503571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d7431fa6-5cae-4b34-9c4c-15667dc6db35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bd54a561aa948888095879de89cd98b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:35.527930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-05f453b0-ce10-47e3-b051-529fe9ac6415 None None] MFA Rules not processed for user `e9f4d8190ae644ae88ac0cf252e850f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:35.541619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-05f453b0-ce10-47e3-b051-529fe9ac6415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:35.577147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 863/1736] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:35 2022] DELETE /identity/v3/users/6bd54a561aa948888095879de89cd98b => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:35.582343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137acb13-fe20-4a0b-83ec-2d4f9e1a801d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:35.582667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137acb13-fe20-4a0b-83ec-2d4f9e1a801d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:35.582951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137acb13-fe20-4a0b-83ec-2d4f9e1a801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:35.627754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-137acb13-fe20-4a0b-83ec-2d4f9e1a801d None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:35.630288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-137acb13-fe20-4a0b-83ec-2d4f9e1a801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:35.761789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 874/1737] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:35.837892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 864/1738] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:35.869015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f453b0-ce10-47e3-b051-529fe9ac6415 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:35.871186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-05f453b0-ce10-47e3-b051-529fe9ac6415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:35.924692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:36.016475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137acb13-fe20-4a0b-83ec-2d4f9e1a801d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:36.017849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-137acb13-fe20-4a0b-83ec-2d4f9e1a801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:36.054903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137acb13-fe20-4a0b-83ec-2d4f9e1a801d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:36.056457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f453b0-ce10-47e3-b051-529fe9ac6415 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:36.075793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:36.077019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:36.077310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:36.077591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] PATH_INFO: `/v3/users/371c1d2225004daba2922154030cf3c1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:36.124509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371c1d2225004daba2922154030cf3c1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:36.126885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:36.227466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b8386-f046-41f5-a68c-06d65d3922e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:36.228297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b8386-f046-41f5-a68c-06d65d3922e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:36.232059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b8386-f046-41f5-a68c-06d65d3922e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:36.232582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b8386-f046-41f5-a68c-06d65d3922e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:36.265744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b8386-f046-41f5-a68c-06d65d3922e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:36.269337 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b8386-f046-41f5-a68c-06d65d3922e9 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:36.283165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371c1d2225004daba2922154030cf3c1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:36.287249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371c1d2225004daba2922154030cf3c1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:36.298814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 875/1739] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:04:36.313949 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371c1d2225004daba2922154030cf3c1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:36.316597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371c1d2225004daba2922154030cf3c1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:36.385359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371c1d2225004daba2922154030cf3c1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:36.479501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 865/1740] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:36 2022] DELETE /identity/v3/users/371c1d2225004daba2922154030cf3c1 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:04:36.484410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177b880-9e96-4075-ad3c-3fccd9f9ac0f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:36.484705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177b880-9e96-4075-ad3c-3fccd9f9ac0f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:36.485338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177b880-9e96-4075-ad3c-3fccd9f9ac0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:36.527198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9177b880-9e96-4075-ad3c-3fccd9f9ac0f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:36.535112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9177b880-9e96-4075-ad3c-3fccd9f9ac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:36.709766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 876/1741] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:36.922570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:36.924871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:37.086852 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:37.109779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632739f-4021-4af8-b0f2-2193012a89b2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:37.110381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632739f-4021-4af8-b0f2-2193012a89b2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:37.110590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632739f-4021-4af8-b0f2-2193012a89b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:37.160013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9df3-e2d6-4cda-9f76-beac11322890 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:37.172012 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6632739f-4021-4af8-b0f2-2193012a89b2 None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:37.173689 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6632739f-4021-4af8-b0f2-2193012a89b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:37.176008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6f394-b61e-483f-866c-838700e5a73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:37.177208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6f394-b61e-483f-866c-838700e5a73b service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:37.177805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6f394-b61e-483f-866c-838700e5a73b service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:37.178416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6f394-b61e-483f-866c-838700e5a73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:37.208189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6f394-b61e-483f-866c-838700e5a73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:37.210937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6f394-b61e-483f-866c-838700e5a73b service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:37.260789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 866/1742] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:37.263098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716860dc-a899-45e1-bf67-17b46aceb475 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:37.268445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716860dc-a899-45e1-bf67-17b46aceb475 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:37.269163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716860dc-a899-45e1-bf67-17b46aceb475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:37.380702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-716860dc-a899-45e1-bf67-17b46aceb475 None None] MFA Rules not processed for user `48e77af2b3a2409fa6d352b0d2674b85`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:37.384997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-716860dc-a899-45e1-bf67-17b46aceb475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:37.419130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 877/1743] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:04:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:37.603953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 867/1744] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:37.745135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632739f-4021-4af8-b0f2-2193012a89b2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:37.746647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6632739f-4021-4af8-b0f2-2193012a89b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:37.819309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:37.923627 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632739f-4021-4af8-b0f2-2193012a89b2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:37.958160 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68990604-61c0-40dc-aea4-d864112079b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:37.962889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68990604-61c0-40dc-aea4-d864112079b8 service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:37.975967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68990604-61c0-40dc-aea4-d864112079b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:37.976478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68990604-61c0-40dc-aea4-d864112079b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:38.012554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68990604-61c0-40dc-aea4-d864112079b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:38.027663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68990604-61c0-40dc-aea4-d864112079b8 service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:38.061543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 878/1745] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:04:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:04:38.919533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716860dc-a899-45e1-bf67-17b46aceb475 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:38.921138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-716860dc-a899-45e1-bf67-17b46aceb475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:38.967321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716860dc-a899-45e1-bf67-17b46aceb475 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:38.985102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:38.986695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:38.987932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:38.988444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] PATH_INFO: `/v3/users/3019e93bb0bb42508d6eb90b9bd44f7e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:38.992983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3019e93bb0bb42508d6eb90b9bd44f7e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:38.996181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:39.180369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3019e93bb0bb42508d6eb90b9bd44f7e'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:39.183203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3019e93bb0bb42508d6eb90b9bd44f7e'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:39.235916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3019e93bb0bb42508d6eb90b9bd44f7e'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:39.236281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3019e93bb0bb42508d6eb90b9bd44f7e'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:39.288791 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3019e93bb0bb42508d6eb90b9bd44f7e'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:39.388607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 868/1746] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:38 2022] DELETE /identity/v3/users/3019e93bb0bb42508d6eb90b9bd44f7e => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:39.392851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3702e-941c-418b-abd0-79afedcd91dc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:39.393485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3702e-941c-418b-abd0-79afedcd91dc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:39.393670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3702e-941c-418b-abd0-79afedcd91dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:39.441234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cea3702e-941c-418b-abd0-79afedcd91dc None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:39.444334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cea3702e-941c-418b-abd0-79afedcd91dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:39.675037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 879/1747] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:39.750052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:39.751935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:39.896533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea699746-a005-487e-b5a2-63b866420fd4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:39.903650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b515ea-9df7-40a4-91fa-50b367895f46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:39.905038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b515ea-9df7-40a4-91fa-50b367895f46 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:39.905038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b515ea-9df7-40a4-91fa-50b367895f46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:39.905038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b515ea-9df7-40a4-91fa-50b367895f46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:39.934528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b515ea-9df7-40a4-91fa-50b367895f46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:39.949575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b515ea-9df7-40a4-91fa-50b367895f46 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:39.955644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:39.969950 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 869/1748] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:40.210704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de34d8-3578-4f26-9bd6-36d09a49ffeb None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:40.214039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de34d8-3578-4f26-9bd6-36d09a49ffeb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:40.214039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de34d8-3578-4f26-9bd6-36d09a49ffeb None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:40.219023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 880/1749] 10.222.0.33 () {54 vars in 912 bytes} [Mon Jun 20 11:04:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:40.226593 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a6a03-ccf4-485d-b7a5-3a619406de77 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:40.227684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a6a03-ccf4-485d-b7a5-3a619406de77 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:40.228077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a6a03-ccf4-485d-b7a5-3a619406de77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:40.306290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4f8a6a03-ccf4-485d-b7a5-3a619406de77 None None] MFA Rules not processed for user `d73fe529c7d342f895aabcb9a39aba78`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:40.312466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4f8a6a03-ccf4-485d-b7a5-3a619406de77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:40.374650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 870/1750] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:04:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:40.443604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de34d8-3578-4f26-9bd6-36d09a49ffeb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:40.463097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de34d8-3578-4f26-9bd6-36d09a49ffeb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:40.483956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976c04f-1eed-4386-a130-09d60ada3377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:40.483956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976c04f-1eed-4386-a130-09d60ada3377 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:40.483956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976c04f-1eed-4386-a130-09d60ada3377 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:40.483956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976c04f-1eed-4386-a130-09d60ada3377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:40.496599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a6a03-ccf4-485d-b7a5-3a619406de77 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:40.499368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4f8a6a03-ccf4-485d-b7a5-3a619406de77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:40.543558 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976c04f-1eed-4386-a130-09d60ada3377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:40.552557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a6a03-ccf4-485d-b7a5-3a619406de77 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:40.553642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976c04f-1eed-4386-a130-09d60ada3377 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:40.578836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:40.583084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:40.587490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:40.587838 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] PATH_INFO: `/v3/projects/91f07f1f9f8c4be59e9bd3e0be4a1ea3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:40.590862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f07f1f9f8c4be59e9bd3e0be4a1ea3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:40.597946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:40.618374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 881/1751] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:40.624734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56a666-4b00-4616-9f0f-1c12902ac3a2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:40.629589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56a666-4b00-4616-9f0f-1c12902ac3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:40.634285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56a666-4b00-4616-9f0f-1c12902ac3a2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:40.643665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 882/1752] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 11:04:40 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:40.768842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec3499-b4c4-4f7b-b159-f1293b643767 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:40.769135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec3499-b4c4-4f7b-b159-f1293b643767 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:40.769588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec3499-b4c4-4f7b-b159-f1293b643767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:40.848511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f07f1f9f8c4be59e9bd3e0be4a1ea3', 'request_id': 'req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:40.849353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-60ec3499-b4c4-4f7b-b159-f1293b643767 None None] MFA Rules not processed for user `bff19e37fa2243fda914f7f4e662c8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:40.850656 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-60ec3499-b4c4-4f7b-b159-f1293b643767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:40.868169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f07f1f9f8c4be59e9bd3e0be4a1ea3', 'request_id': 'req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:40.886553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f07f1f9f8c4be59e9bd3e0be4a1ea3', 'request_id': 'req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:40.928350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] The token cache is being invalidate because project 91f07f1f9f8c4be59e9bd3e0be4a1ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:40.929939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:40.933543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:40.939373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 871/1753] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:40 2022] DELETE /identity/v3/projects/91f07f1f9f8c4be59e9bd3e0be4a1ea3 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:04:40.944636 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:40.951492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:41.180563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 883/1754] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 11:04:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 20 11:04:41.270445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda19655-20c2-4cef-9bd8-07bc4bec6fa3 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:41.298212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:41.299633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:41.300060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:41.300461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] PATH_INFO: `/v3/users/aa5c0b89e3d84736b1478610da6c0e57` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:41.306519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5c0b89e3d84736b1478610da6c0e57)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:41.309575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:41.465868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5c0b89e3d84736b1478610da6c0e57'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:41.467435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5c0b89e3d84736b1478610da6c0e57'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:41.487360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5c0b89e3d84736b1478610da6c0e57'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:41.488108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5c0b89e3d84736b1478610da6c0e57'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:41.505893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5c0b89e3d84736b1478610da6c0e57'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:41.544909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:41.593597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 872/1755] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:40 2022] DELETE /identity/v3/users/aa5c0b89e3d84736b1478610da6c0e57 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:41.748926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec3499-b4c4-4f7b-b159-f1293b643767 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:41.750711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-60ec3499-b4c4-4f7b-b159-f1293b643767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:41.798701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec3499-b4c4-4f7b-b159-f1293b643767 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:41.859448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:41.861587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:41.935473 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:42.056053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8cc2b-505e-4d19-a562-d3487e7a607f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:42.059358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8cc2b-505e-4d19-a562-d3487e7a607f service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:42.059676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8cc2b-505e-4d19-a562-d3487e7a607f service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:42.059959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8cc2b-505e-4d19-a562-d3487e7a607f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:42.068117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8af8cc2b-505e-4d19-a562-d3487e7a607f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:42.114595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8cc2b-505e-4d19-a562-d3487e7a607f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:42.118277 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8cc2b-505e-4d19-a562-d3487e7a607f service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:42.141645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fc92e-60fc-4a33-9da8-341c3093cd72 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:42.171680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 884/1756] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:42.186995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:42.189413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:42.189712 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:42.190057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] PATH_INFO: `/v3/projects/c371c880db814276ac724b8d620c878e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:42.196434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c371c880db814276ac724b8d620c878e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:42.208543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:42.399556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c371c880db814276ac724b8d620c878e', 'request_id': 'req-d4bd88fe-233b-45ac-aded-5810e158ca9a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:42.409620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c371c880db814276ac724b8d620c878e', 'request_id': 'req-d4bd88fe-233b-45ac-aded-5810e158ca9a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:42.434330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c371c880db814276ac724b8d620c878e', 'request_id': 'req-d4bd88fe-233b-45ac-aded-5810e158ca9a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:42.468633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] The token cache is being invalidate because project c371c880db814276ac724b8d620c878e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:42.468931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:42.477307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d4bd88fe-233b-45ac-aded-5810e158ca9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:42.484223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 873/1757] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:41 2022] DELETE /identity/v3/projects/c371c880db814276ac724b8d620c878e => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:42.485849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8cc2b-505e-4d19-a562-d3487e7a607f service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:42.503474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8cc2b-505e-4d19-a562-d3487e7a607f service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:42.708681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:42.712540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:42.714232 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:42.715465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] PATH_INFO: `/v3/users/bcc5cc4402d9406eabaed6804ad33c3d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:42.718595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc5cc4402d9406eabaed6804ad33c3d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:42.720659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:42.928231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc5cc4402d9406eabaed6804ad33c3d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:42.952943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc5cc4402d9406eabaed6804ad33c3d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:42.955108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc5cc4402d9406eabaed6804ad33c3d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:42.958190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc5cc4402d9406eabaed6804ad33c3d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:43.022634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc5cc4402d9406eabaed6804ad33c3d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:43.100531 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 885/1758] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:42 2022] DELETE /identity/v3/users/bcc5cc4402d9406eabaed6804ad33c3d => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:43.552028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72cc64-271d-44a5-bfd4-415052e32d5c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:43.558470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72cc64-271d-44a5-bfd4-415052e32d5c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:43.558470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72cc64-271d-44a5-bfd4-415052e32d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:43.607069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-aa72cc64-271d-44a5-bfd4-415052e32d5c None None] MFA Rules not processed for user `d73fe529c7d342f895aabcb9a39aba78`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:43.608532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-aa72cc64-271d-44a5-bfd4-415052e32d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:43.697369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 874/1759] 10.222.0.44 () {58 vars in 1015 bytes} [Mon Jun 20 11:04:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:43.751690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:43.763408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:43.797845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1371bc8-c351-4f54-a905-771f944d73dd admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:43.809917 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7880b8-ade6-459f-a3f0-43cf54f8a3b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:43.810676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7880b8-ade6-459f-a3f0-43cf54f8a3b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:43.810972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7880b8-ade6-459f-a3f0-43cf54f8a3b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:43.811249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7880b8-ade6-459f-a3f0-43cf54f8a3b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:43.822312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7880b8-ade6-459f-a3f0-43cf54f8a3b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:43.830570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7880b8-ade6-459f-a3f0-43cf54f8a3b1 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:43.842073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 886/1760] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:43.872584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72cc64-271d-44a5-bfd4-415052e32d5c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:43.883924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-aa72cc64-271d-44a5-bfd4-415052e32d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:44.003062 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:44.096526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72cc64-271d-44a5-bfd4-415052e32d5c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:44.126736 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:44.129860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:44.130493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:44.138505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] PATH_INFO: `/v3/projects/d40d4d8339dc408fb620ada67f2ed9e3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:44.142377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d40d4d8339dc408fb620ada67f2ed9e3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:44.145845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:44.290121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd40d4d8339dc408fb620ada67f2ed9e3', 'request_id': 'req-266de274-6360-4afd-9f0c-4bc8e52ab900'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:44.303449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd40d4d8339dc408fb620ada67f2ed9e3', 'request_id': 'req-266de274-6360-4afd-9f0c-4bc8e52ab900'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:44.311853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd40d4d8339dc408fb620ada67f2ed9e3', 'request_id': 'req-266de274-6360-4afd-9f0c-4bc8e52ab900'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:44.327160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] The token cache is being invalidate because project d40d4d8339dc408fb620ada67f2ed9e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:44.327709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:44.331389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-266de274-6360-4afd-9f0c-4bc8e52ab900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:44.333586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 875/1761] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:43 2022] DELETE /identity/v3/projects/d40d4d8339dc408fb620ada67f2ed9e3 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:44.338961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208be53-fa81-4c3b-962a-c571fc37cfc1 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:44.339502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208be53-fa81-4c3b-962a-c571fc37cfc1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:44.339502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1208be53-fa81-4c3b-962a-c571fc37cfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:44.380887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1208be53-fa81-4c3b-962a-c571fc37cfc1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:44.382548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1208be53-fa81-4c3b-962a-c571fc37cfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:44.429390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:44.494139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 887/1762] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:44.504818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f3a9a-221e-483e-a93e-149662725435 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:44.505115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f3a9a-221e-483e-a93e-149662725435 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:44.505406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f3a9a-221e-483e-a93e-149662725435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:44.559763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9e0f3a9a-221e-483e-a93e-149662725435 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:44.572545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9e0f3a9a-221e-483e-a93e-149662725435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:44.584472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 876/1763] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:44.588105 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208be53-fa81-4c3b-962a-c571fc37cfc1 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:44.598712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1208be53-fa81-4c3b-962a-c571fc37cfc1 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:44.620073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a31b3f-9461-46cb-92e2-dad3eaef3bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:44.620777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a31b3f-9461-46cb-92e2-dad3eaef3bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:44.621066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a31b3f-9461-46cb-92e2-dad3eaef3bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:44.621351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a31b3f-9461-46cb-92e2-dad3eaef3bdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:44.622319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a31b3f-9461-46cb-92e2-dad3eaef3bdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:44.622638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a31b3f-9461-46cb-92e2-dad3eaef3bdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:44.624297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a31b3f-9461-46cb-92e2-dad3eaef3bdf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:44.646363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f3a9a-221e-483e-a93e-149662725435 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:44.647858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9e0f3a9a-221e-483e-a93e-149662725435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:44.675950 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f3a9a-221e-483e-a93e-149662725435 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:44.676947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 888/1764] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:44.677892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a31b3f-9461-46cb-92e2-dad3eaef3bdf admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:44.696379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:44.697283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:44.697568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:44.697849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] PATH_INFO: `/v3/projects/013a9ec3d79745be8736fe2cb7d2a9c1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:44.702150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=013a9ec3d79745be8736fe2cb7d2a9c1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:44.704175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:44.716818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a31b3f-9461-46cb-92e2-dad3eaef3bdf admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:44.737529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22cead-a320-40e0-81a0-4a788e626789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:44.738167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22cead-a320-40e0-81a0-4a788e626789 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:44.738481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22cead-a320-40e0-81a0-4a788e626789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:44.738733 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22cead-a320-40e0-81a0-4a788e626789 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:44.739448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22cead-a320-40e0-81a0-4a788e626789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:44.742038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22cead-a320-40e0-81a0-4a788e626789 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:44.882293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '013a9ec3d79745be8736fe2cb7d2a9c1', 'request_id': 'req-793e9fd0-36aa-4f92-bcde-2281521a9dfe'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:44.891044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 889/1765] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:44 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:44.898988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '013a9ec3d79745be8736fe2cb7d2a9c1', 'request_id': 'req-793e9fd0-36aa-4f92-bcde-2281521a9dfe'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:44.902200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22cead-a320-40e0-81a0-4a788e626789 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:44.908474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '013a9ec3d79745be8736fe2cb7d2a9c1', 'request_id': 'req-793e9fd0-36aa-4f92-bcde-2281521a9dfe'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:44.948523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] The token cache is being invalidate because project 013a9ec3d79745be8736fe2cb7d2a9c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:44.948769 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:44.952077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-793e9fd0-36aa-4f92-bcde-2281521a9dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:44.953971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 877/1766] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:44 2022] DELETE /identity/v3/projects/013a9ec3d79745be8736fe2cb7d2a9c1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:44.959276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769acf4-f575-4c17-ba28-d0fb16680fe6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:44.959569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769acf4-f575-4c17-ba28-d0fb16680fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:44.959856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769acf4-f575-4c17-ba28-d0fb16680fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.018673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a769acf4-f575-4c17-ba28-d0fb16680fe6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:45.020176 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a769acf4-f575-4c17-ba28-d0fb16680fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:45.082848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:45.128194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22cead-a320-40e0-81a0-4a788e626789 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:45.143957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b343aa-fa08-4f60-b285-892159ec378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:45.148823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b343aa-fa08-4f60-b285-892159ec378e admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.149378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b343aa-fa08-4f60-b285-892159ec378e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.149859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b343aa-fa08-4f60-b285-892159ec378e admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.153952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b343aa-fa08-4f60-b285-892159ec378e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:45.156421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 878/1767] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:45.158404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b343aa-fa08-4f60-b285-892159ec378e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:45.167405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa85ac-60bc-4fe6-a41f-a66b23a2372f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.170570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa85ac-60bc-4fe6-a41f-a66b23a2372f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.173006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa85ac-60bc-4fe6-a41f-a66b23a2372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.202967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 890/1768] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:44 2022] POST /identity/v3/users => generated 287 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:45.207168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b343aa-fa08-4f60-b285-892159ec378e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:45.213171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-86b343aa-fa08-4f60-b285-892159ec378e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:45.230735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-65aa85ac-60bc-4fe6-a41f-a66b23a2372f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:45.234959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-65aa85ac-60bc-4fe6-a41f-a66b23a2372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:45.238598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b343aa-fa08-4f60-b285-892159ec378e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:45.251117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a30596-6303-4465-86cf-55ab571a62f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:45.251815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a30596-6303-4465-86cf-55ab571a62f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.262792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 879/1769] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:45.262982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a30596-6303-4465-86cf-55ab571a62f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.262982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a30596-6303-4465-86cf-55ab571a62f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.262982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a30596-6303-4465-86cf-55ab571a62f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:45.265972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa85ac-60bc-4fe6-a41f-a66b23a2372f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:45.271463 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a30596-6303-4465-86cf-55ab571a62f6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:45.291250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa85ac-60bc-4fe6-a41f-a66b23a2372f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:45.305462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 891/1770] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:45.305804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd524d0e-c2da-401e-973d-47c1b0b38a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:45.306536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd524d0e-c2da-401e-973d-47c1b0b38a15 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.306830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd524d0e-c2da-401e-973d-47c1b0b38a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.307120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd524d0e-c2da-401e-973d-47c1b0b38a15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.308056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd524d0e-c2da-401e-973d-47c1b0b38a15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:45.308409 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd524d0e-c2da-401e-973d-47c1b0b38a15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:45.312871 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a30596-6303-4465-86cf-55ab571a62f6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:45.313870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd524d0e-c2da-401e-973d-47c1b0b38a15 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:45.331615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 880/1771] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:45.344114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd524d0e-c2da-401e-973d-47c1b0b38a15 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:45.352535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a30596-6303-4465-86cf-55ab571a62f6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:45.364866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bb6e2-03a7-4984-94d8-99e71eeeeed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:45.367027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bb6e2-03a7-4984-94d8-99e71eeeeed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.367154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bb6e2-03a7-4984-94d8-99e71eeeeed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.367384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bb6e2-03a7-4984-94d8-99e71eeeeed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.368127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bb6e2-03a7-4984-94d8-99e71eeeeed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:45.373472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd524d0e-c2da-401e-973d-47c1b0b38a15 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:45.374489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bb6e2-03a7-4984-94d8-99e71eeeeed1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:45.403200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41919513-32bb-4757-ac0a-33b500a98123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:45.403947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41919513-32bb-4757-ac0a-33b500a98123 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.404243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41919513-32bb-4757-ac0a-33b500a98123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.404529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41919513-32bb-4757-ac0a-33b500a98123 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.405258 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41919513-32bb-4757-ac0a-33b500a98123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:45.407167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 892/1772] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:45.407406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41919513-32bb-4757-ac0a-33b500a98123 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:45.412649 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bb6e2-03a7-4984-94d8-99e71eeeeed1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:45.440626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bb6e2-03a7-4984-94d8-99e71eeeeed1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:45.447727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4f3a1-4e28-446b-948b-63df368a4f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:45.449004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4f3a1-4e28-446b-948b-63df368a4f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.449293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4f3a1-4e28-446b-948b-63df368a4f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.449575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4f3a1-4e28-446b-948b-63df368a4f90 admin admin] PATH_INFO: `/v3/projects/c01607c0fc0b4090a85320414d1def1f/users/4f70fa53427c4eafb06aef29f04d0b5e/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.494527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4f3a1-4e28-446b-948b-63df368a4f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c01607c0fc0b4090a85320414d1def1f, user_id=4f70fa53427c4eafb06aef29f04d0b5e, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:45.508027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 881/1773] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:45 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:45.508256 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4f3a1-4e28-446b-948b-63df368a4f90 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:45.513243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41919513-32bb-4757-ac0a-33b500a98123 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:45.562050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 893/1774] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:45 2022] PUT /identity/v3/projects/c01607c0fc0b4090a85320414d1def1f/users/4f70fa53427c4eafb06aef29f04d0b5e/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:45.573898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece1b1e-9ddd-480e-9a3a-b56740037d4e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.574646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece1b1e-9ddd-480e-9a3a-b56740037d4e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.574646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece1b1e-9ddd-480e-9a3a-b56740037d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.641312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41919513-32bb-4757-ac0a-33b500a98123 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:45.647937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-aece1b1e-9ddd-480e-9a3a-b56740037d4e None None] MFA Rules not processed for user `4f70fa53427c4eafb06aef29f04d0b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:45.652410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-aece1b1e-9ddd-480e-9a3a-b56740037d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:45.714600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:45.796450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805af1e-fdaa-4191-9440-8eb1248d3f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:45.798508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805af1e-fdaa-4191-9440-8eb1248d3f02 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.798994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805af1e-fdaa-4191-9440-8eb1248d3f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.799737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805af1e-fdaa-4191-9440-8eb1248d3f02 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.800639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805af1e-fdaa-4191-9440-8eb1248d3f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:45.803865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805af1e-fdaa-4191-9440-8eb1248d3f02 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:45.830799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 894/1775] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:04:45.835979 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece1b1e-9ddd-480e-9a3a-b56740037d4e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:45.852511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 882/1776] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:45 2022] POST /identity/v3/users => generated 294 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:45.859324 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805af1e-fdaa-4191-9440-8eb1248d3f02 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:45.874742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece1b1e-9ddd-480e-9a3a-b56740037d4e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:45.878071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805af1e-fdaa-4191-9440-8eb1248d3f02 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:45.884787 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f78ff-5170-4852-beb6-86c293757aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:45.885932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f78ff-5170-4852-beb6-86c293757aeb admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.886536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f78ff-5170-4852-beb6-86c293757aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.889884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f78ff-5170-4852-beb6-86c293757aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.891307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f78ff-5170-4852-beb6-86c293757aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:45.893003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f78ff-5170-4852-beb6-86c293757aeb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:45.903336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcc2c3-c0d0-4e34-a60b-71a9cabd74f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:45.904045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcc2c3-c0d0-4e34-a60b-71a9cabd74f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.904333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcc2c3-c0d0-4e34-a60b-71a9cabd74f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.904625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcc2c3-c0d0-4e34-a60b-71a9cabd74f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.905331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcc2c3-c0d0-4e34-a60b-71a9cabd74f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:45.907300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcc2c3-c0d0-4e34-a60b-71a9cabd74f9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:45.939726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 883/1777] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:04:45.942340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f78ff-5170-4852-beb6-86c293757aeb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:45.968634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f78ff-5170-4852-beb6-86c293757aeb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:45.976764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c8a77-1fa6-4a0b-b981-1193c92389b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:45.977559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c8a77-1fa6-4a0b-b981-1193c92389b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:45.978012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c8a77-1fa6-4a0b-b981-1193c92389b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:45.978563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c8a77-1fa6-4a0b-b981-1193c92389b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:45.979896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c8a77-1fa6-4a0b-b981-1193c92389b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:45.982234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c8a77-1fa6-4a0b-b981-1193c92389b5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:46.000175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 884/1778] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:04:46.001665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c8a77-1fa6-4a0b-b981-1193c92389b5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:46.018415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 895/1779] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:45 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:46.021928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcc2c3-c0d0-4e34-a60b-71a9cabd74f9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:46.024102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c8a77-1fa6-4a0b-b981-1193c92389b5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:46.049178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:46.196389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5dfb6-b0f9-4e2a-8511-02bc76285a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:46.197348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5dfb6-b0f9-4e2a-8511-02bc76285a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:46.197656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5dfb6-b0f9-4e2a-8511-02bc76285a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:46.198208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5dfb6-b0f9-4e2a-8511-02bc76285a46 admin admin] PATH_INFO: `/v3/projects/379971362a734eff897adfe04a7d2cdd/users/e37438f1264e4c4bbda467d962c66f62/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:46.200855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcc2c3-c0d0-4e34-a60b-71a9cabd74f9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:46.219210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de5f30-0ce0-4e01-8821-8bb598ba1644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:46.219860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5f30-0ce0-4e01-8821-8bb598ba1644 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:46.220146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5f30-0ce0-4e01-8821-8bb598ba1644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:46.220449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5f30-0ce0-4e01-8821-8bb598ba1644 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:46.221192 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de5f30-0ce0-4e01-8821-8bb598ba1644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:46.231589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de5f30-0ce0-4e01-8821-8bb598ba1644 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:46.263529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5dfb6-b0f9-4e2a-8511-02bc76285a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=379971362a734eff897adfe04a7d2cdd, user_id=e37438f1264e4c4bbda467d962c66f62, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:46.289042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5dfb6-b0f9-4e2a-8511-02bc76285a46 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:46.313792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 896/1780] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:46 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:46.315138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de5f30-0ce0-4e01-8821-8bb598ba1644 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:46.325921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 885/1781] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:46 2022] PUT /identity/v3/projects/379971362a734eff897adfe04a7d2cdd/users/e37438f1264e4c4bbda467d962c66f62/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:46.338258 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b01970-20ac-4cc7-a3e8-b5c1afcdcc31 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:46.339471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b01970-20ac-4cc7-a3e8-b5c1afcdcc31 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:46.339471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b01970-20ac-4cc7-a3e8-b5c1afcdcc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:46.343563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de5f30-0ce0-4e01-8821-8bb598ba1644 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:46.380682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-41b01970-20ac-4cc7-a3e8-b5c1afcdcc31 None None] MFA Rules not processed for user `e37438f1264e4c4bbda467d962c66f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:46.382445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-41b01970-20ac-4cc7-a3e8-b5c1afcdcc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:46.499781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd8901-1eb7-48c5-bc55-69690fb8baba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:46.500483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd8901-1eb7-48c5-bc55-69690fb8baba admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:46.500772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd8901-1eb7-48c5-bc55-69690fb8baba admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:46.501049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd8901-1eb7-48c5-bc55-69690fb8baba admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:46.501786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd8901-1eb7-48c5-bc55-69690fb8baba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:46.503617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd8901-1eb7-48c5-bc55-69690fb8baba admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:46.540852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 897/1782] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:46.540852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c5bd3-aede-4de2-94e5-a975c5c838e3 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:46.540852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c5bd3-aede-4de2-94e5-a975c5c838e3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:46.540852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c5bd3-aede-4de2-94e5-a975c5c838e3 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:46.550587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 898/1783] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 11:04:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:46.555702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c5bd3-aede-4de2-94e5-a975c5c838e3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:46.557913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 886/1784] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:46.565266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b01970-20ac-4cc7-a3e8-b5c1afcdcc31 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:46.599503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c5bd3-aede-4de2-94e5-a975c5c838e3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:46.600161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b01970-20ac-4cc7-a3e8-b5c1afcdcc31 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:46.618544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7cf38-2f55-4511-bc93-955f45ad0a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:46.619535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75523bc7-14cb-4f08-85cc-f8c25b23fa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:46.624538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7cf38-2f55-4511-bc93-955f45ad0a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:46.625674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75523bc7-14cb-4f08-85cc-f8c25b23fa82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:46.626016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7cf38-2f55-4511-bc93-955f45ad0a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:46.626434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7cf38-2f55-4511-bc93-955f45ad0a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:46.626819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75523bc7-14cb-4f08-85cc-f8c25b23fa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:46.627661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7cf38-2f55-4511-bc93-955f45ad0a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:46.628322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75523bc7-14cb-4f08-85cc-f8c25b23fa82 admin admin] PATH_INFO: `/v3/projects/60914d0ecf294ea79fbc4a7dfcc6db2b/users/57a75ad0773e474281fb139fcb72e6af/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:46.631407 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7cf38-2f55-4511-bc93-955f45ad0a19 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:46.704690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75523bc7-14cb-4f08-85cc-f8c25b23fa82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60914d0ecf294ea79fbc4a7dfcc6db2b, user_id=57a75ad0773e474281fb139fcb72e6af, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:46.708668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75523bc7-14cb-4f08-85cc-f8c25b23fa82 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:46.743013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 887/1785] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:46 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:46.743689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba202b4-11ae-4121-a9b5-00b06e8a1571 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:46.746076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba202b4-11ae-4121-a9b5-00b06e8a1571 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:46.749404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba202b4-11ae-4121-a9b5-00b06e8a1571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:46.754015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 899/1786] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:46 2022] PUT /identity/v3/projects/60914d0ecf294ea79fbc4a7dfcc6db2b/users/57a75ad0773e474281fb139fcb72e6af/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:46.755028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75523bc7-14cb-4f08-85cc-f8c25b23fa82 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:46.764512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-75523bc7-14cb-4f08-85cc-f8c25b23fa82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:46.792972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:46.815077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-fba202b4-11ae-4121-a9b5-00b06e8a1571 None None] MFA Rules not processed for user `bff19e37fa2243fda914f7f4e662c8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:46.816495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fba202b4-11ae-4121-a9b5-00b06e8a1571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:47.000885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75523bc7-14cb-4f08-85cc-f8c25b23fa82 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:47.084522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:47.113789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 888/1787] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:47.115554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba202b4-11ae-4121-a9b5-00b06e8a1571 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:47.120313 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fba202b4-11ae-4121-a9b5-00b06e8a1571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:47.180376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:47.181561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:47.182008 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:47.182510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] PATH_INFO: `/v3/users/b9ced524933d4bcf9da27c0a2aaf3175` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:47.188747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9ced524933d4bcf9da27c0a2aaf3175)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:47.199362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:47.294839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9ced524933d4bcf9da27c0a2aaf3175'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:47.314148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9ced524933d4bcf9da27c0a2aaf3175'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:47.315879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9ced524933d4bcf9da27c0a2aaf3175'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:47.317632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9ced524933d4bcf9da27c0a2aaf3175'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:47.338935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba202b4-11ae-4121-a9b5-00b06e8a1571 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:47.357210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9ced524933d4bcf9da27c0a2aaf3175'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:47.358793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:47.359711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:47.364099 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:47.364552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] PATH_INFO: `/v3/projects/a0f2168568bb424d87f528585f4a576a` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:47.366878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f2168568bb424d87f528585f4a576a)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:47.374912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:47.395832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 900/1788] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:46 2022] DELETE /identity/v3/users/b9ced524933d4bcf9da27c0a2aaf3175 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:47.396723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:47.402807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:47.484439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:47.516036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f2168568bb424d87f528585f4a576a', 'request_id': 'req-e6c16684-e85b-465e-861b-3d156852012f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:47.535724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f2168568bb424d87f528585f4a576a', 'request_id': 'req-e6c16684-e85b-465e-861b-3d156852012f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:47.552050 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f2168568bb424d87f528585f4a576a', 'request_id': 'req-e6c16684-e85b-465e-861b-3d156852012f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:47.553352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2166fc2-80ea-4456-becf-8fc1ad8bfddf admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:47.592514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] The token cache is being invalidate because project a0f2168568bb424d87f528585f4a576a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:47.598168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:47.599198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:47.601156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 889/1789] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:47 2022] DELETE /identity/v3/projects/a0f2168568bb424d87f528585f4a576a => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:47.604513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:47.605018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:47.742844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b2835-f272-48ab-b8e0-3290a336359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:47.744642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2835-f272-48ab-b8e0-3290a336359b admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:47.744960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2835-f272-48ab-b8e0-3290a336359b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:47.745280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b2835-f272-48ab-b8e0-3290a336359b admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:47.745957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b2835-f272-48ab-b8e0-3290a336359b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:47.748039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b2835-f272-48ab-b8e0-3290a336359b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:47.816286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 901/1790] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:47 2022] POST /identity/v3/users => generated 299 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:47.817401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b2835-f272-48ab-b8e0-3290a336359b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:47.822315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c16684-e85b-465e-861b-3d156852012f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:47.829164 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0e0b2835-f272-48ab-b8e0-3290a336359b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:47.847937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d115e7-821d-499c-bb38-44f92a783fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:47.849003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d115e7-821d-499c-bb38-44f92a783fef admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:47.849003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d115e7-821d-499c-bb38-44f92a783fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:47.849146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d115e7-821d-499c-bb38-44f92a783fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:47.849887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d115e7-821d-499c-bb38-44f92a783fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:47.853959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d115e7-821d-499c-bb38-44f92a783fef admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:47.894739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 890/1791] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:47.911132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a11a7-5b2d-445b-b5ff-844704428895 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:47.914722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a11a7-5b2d-445b-b5ff-844704428895 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:47.918798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a11a7-5b2d-445b-b5ff-844704428895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:47.977064 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-137a11a7-5b2d-445b-b5ff-844704428895 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:47.978336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-137a11a7-5b2d-445b-b5ff-844704428895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:48.007057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 891/1792] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:48.008138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a11a7-5b2d-445b-b5ff-844704428895 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:48.012551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-137a11a7-5b2d-445b-b5ff-844704428895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:48.056154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a11a7-5b2d-445b-b5ff-844704428895 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:48.066703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5527d4a-b3c5-43c8-b1fa-c0b12a812457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:48.067682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5527d4a-b3c5-43c8-b1fa-c0b12a812457 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.067682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5527d4a-b3c5-43c8-b1fa-c0b12a812457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.067918 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5527d4a-b3c5-43c8-b1fa-c0b12a812457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.068662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5527d4a-b3c5-43c8-b1fa-c0b12a812457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:48.070694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5527d4a-b3c5-43c8-b1fa-c0b12a812457 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:48.085363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b2835-f272-48ab-b8e0-3290a336359b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:48.092049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568cd9b-6114-4e07-aa24-26b351b8dc0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:48.093782 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568cd9b-6114-4e07-aa24-26b351b8dc0a service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.094263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568cd9b-6114-4e07-aa24-26b351b8dc0a service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.094743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568cd9b-6114-4e07-aa24-26b351b8dc0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.096494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 892/1793] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:48.097328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0568cd9b-6114-4e07-aa24-26b351b8dc0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:48.097438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5527d4a-b3c5-43c8-b1fa-c0b12a812457 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:48.113721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568cd9b-6114-4e07-aa24-26b351b8dc0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:48.115935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568cd9b-6114-4e07-aa24-26b351b8dc0a service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:48.124487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5527d4a-b3c5-43c8-b1fa-c0b12a812457 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:48.129757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 902/1794] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:48.131225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c595702-e2d9-429a-8e94-c0ca1b4d3075 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.131605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c595702-e2d9-429a-8e94-c0ca1b4d3075 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.131972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c595702-e2d9-429a-8e94-c0ca1b4d3075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.133613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77741ddd-fd82-4ec2-9b97-151ca1f96f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:48.134623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77741ddd-fd82-4ec2-9b97-151ca1f96f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.134911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77741ddd-fd82-4ec2-9b97-151ca1f96f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.135189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77741ddd-fd82-4ec2-9b97-151ca1f96f6b admin admin] PATH_INFO: `/v3/projects/60914d0ecf294ea79fbc4a7dfcc6db2b/users/57a75ad0773e474281fb139fcb72e6af/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.139618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77741ddd-fd82-4ec2-9b97-151ca1f96f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60914d0ecf294ea79fbc4a7dfcc6db2b, user_id=57a75ad0773e474281fb139fcb72e6af, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:48.148038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77741ddd-fd82-4ec2-9b97-151ca1f96f6b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:48.158010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9c595702-e2d9-429a-8e94-c0ca1b4d3075 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:48.159272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9c595702-e2d9-429a-8e94-c0ca1b4d3075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:48.161819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:48.179274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 903/1795] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:48.183275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751ecad-1227-436b-a442-7b1adacf67ab None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.183563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751ecad-1227-436b-a442-7b1adacf67ab None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.183863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 893/1796] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:48 2022] PUT /identity/v3/projects/60914d0ecf294ea79fbc4a7dfcc6db2b/users/57a75ad0773e474281fb139fcb72e6af/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:48.184804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77741ddd-fd82-4ec2-9b97-151ca1f96f6b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:48.194330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751ecad-1227-436b-a442-7b1adacf67ab None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.197029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 904/1797] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 11:04:48 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:48.198018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751ecad-1227-436b-a442-7b1adacf67ab None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:48.273739 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:48.301446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751ecad-1227-436b-a442-7b1adacf67ab None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:48.307485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77741ddd-fd82-4ec2-9b97-151ca1f96f6b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:48.310465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573700a-aa2d-4f95-8142-1d9b4509764b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:48.311665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573700a-aa2d-4f95-8142-1d9b4509764b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.311968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573700a-aa2d-4f95-8142-1d9b4509764b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.312238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573700a-aa2d-4f95-8142-1d9b4509764b admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.313921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573700a-aa2d-4f95-8142-1d9b4509764b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:48.313921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573700a-aa2d-4f95-8142-1d9b4509764b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:48.315981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573700a-aa2d-4f95-8142-1d9b4509764b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:48.321224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195036af-46c7-4e98-84b1-ce4e799c9c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:48.322123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195036af-46c7-4e98-84b1-ce4e799c9c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.322403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195036af-46c7-4e98-84b1-ce4e799c9c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.322683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195036af-46c7-4e98-84b1-ce4e799c9c6c admin admin] PATH_INFO: `/v3/projects/6bcd924495314b99b99bb520a41202bb/users/708f229ff00a4f50bbb51faaeca65556/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.343384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 905/1798] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:48.345829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7984b-e185-4e91-8aa5-150ff262572d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.346356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7984b-e185-4e91-8aa5-150ff262572d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.346838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7984b-e185-4e91-8aa5-150ff262572d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.362621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195036af-46c7-4e98-84b1-ce4e799c9c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bcd924495314b99b99bb520a41202bb, user_id=708f229ff00a4f50bbb51faaeca65556, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:48.365310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195036af-46c7-4e98-84b1-ce4e799c9c6c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:48.393176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2fc7984b-e185-4e91-8aa5-150ff262572d None None] MFA Rules not processed for user `57a75ad0773e474281fb139fcb72e6af`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:48.397328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 894/1799] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:48 2022] PUT /identity/v3/projects/6bcd924495314b99b99bb520a41202bb/users/708f229ff00a4f50bbb51faaeca65556/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:48.397328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195036af-46c7-4e98-84b1-ce4e799c9c6c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:48.398056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2fc7984b-e185-4e91-8aa5-150ff262572d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:48.553858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195036af-46c7-4e98-84b1-ce4e799c9c6c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:48.559153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84e492-090c-4203-8dff-bae16673e2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:48.559854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84e492-090c-4203-8dff-bae16673e2be admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.560144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84e492-090c-4203-8dff-bae16673e2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.560421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84e492-090c-4203-8dff-bae16673e2be admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.561133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84e492-090c-4203-8dff-bae16673e2be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:48.563122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84e492-090c-4203-8dff-bae16673e2be admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:48.581943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 906/1800] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:48.591951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7984b-e185-4e91-8aa5-150ff262572d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:48.612892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7984b-e185-4e91-8aa5-150ff262572d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:48.622615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70610d3-cf9f-48bf-b421-ec36c1675598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:48.623479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70610d3-cf9f-48bf-b421-ec36c1675598 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.623765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70610d3-cf9f-48bf-b421-ec36c1675598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.624309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70610d3-cf9f-48bf-b421-ec36c1675598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.625196 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70610d3-cf9f-48bf-b421-ec36c1675598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:48.628241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70610d3-cf9f-48bf-b421-ec36c1675598 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:48.654845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 907/1801] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:48.654845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930759-2766-43df-9026-1df8ae69ce8a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.659034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930759-2766-43df-9026-1df8ae69ce8a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.659487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98930759-2766-43df-9026-1df8ae69ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.662421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 895/1802] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:48 2022] POST /identity/v3/projects => generated 377 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:48.663393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84e492-090c-4203-8dff-bae16673e2be admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:48.676079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cd84e492-090c-4203-8dff-bae16673e2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:48.701213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-98930759-2766-43df-9026-1df8ae69ce8a None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:48.706263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-98930759-2766-43df-9026-1df8ae69ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:48.803715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84e492-090c-4203-8dff-bae16673e2be admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:48.808443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e645cb2-74b7-49ac-9016-0c152aaa288d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:48.809122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e645cb2-74b7-49ac-9016-0c152aaa288d service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:48.809415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e645cb2-74b7-49ac-9016-0c152aaa288d service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:48.809664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e645cb2-74b7-49ac-9016-0c152aaa288d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:48.852495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:48.899302 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 908/1803] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 11:04:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:48.900316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98930759-2766-43df-9026-1df8ae69ce8a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:48.976173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e645cb2-74b7-49ac-9016-0c152aaa288d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:48.977833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e645cb2-74b7-49ac-9016-0c152aaa288d service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:49.036236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 896/1804] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:49.052589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e645cb2-74b7-49ac-9016-0c152aaa288d service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:49.056039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:49.085069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98930759-2766-43df-9026-1df8ae69ce8a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:49.115998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a17ecf-7b1f-4297-86b9-a654cb301bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:49.117215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a17ecf-7b1f-4297-86b9-a654cb301bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:49.129835 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a17ecf-7b1f-4297-86b9-a654cb301bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:49.130451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a17ecf-7b1f-4297-86b9-a654cb301bb2 admin admin] PATH_INFO: `/v3/projects/6bcd924495314b99b99bb520a41202bb/users/708f229ff00a4f50bbb51faaeca65556/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:49.137969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a17ecf-7b1f-4297-86b9-a654cb301bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bcd924495314b99b99bb520a41202bb, user_id=708f229ff00a4f50bbb51faaeca65556, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:49.148839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a17ecf-7b1f-4297-86b9-a654cb301bb2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:49.196728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 909/1805] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:48 2022] PUT /identity/v3/projects/6bcd924495314b99b99bb520a41202bb/users/708f229ff00a4f50bbb51faaeca65556/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:49.208538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a455a-92d2-4b93-bfd9-c057329442e8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:49.208821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a455a-92d2-4b93-bfd9-c057329442e8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:49.209202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a455a-92d2-4b93-bfd9-c057329442e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:49.306746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b13a455a-92d2-4b93-bfd9-c057329442e8 None None] MFA Rules not processed for user `708f229ff00a4f50bbb51faaeca65556`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:49.308752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b13a455a-92d2-4b93-bfd9-c057329442e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:49.417846 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e645cb2-74b7-49ac-9016-0c152aaa288d service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:49.588610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 910/1806] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:04:49.596535 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa7631-7c27-4d9f-8279-895d331f2a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:49.597801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa7631-7c27-4d9f-8279-895d331f2a7b admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:49.598364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa7631-7c27-4d9f-8279-895d331f2a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:49.598481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a455a-92d2-4b93-bfd9-c057329442e8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:49.598968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa7631-7c27-4d9f-8279-895d331f2a7b admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:49.600292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa7631-7c27-4d9f-8279-895d331f2a7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:49.602217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b13a455a-92d2-4b93-bfd9-c057329442e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:49.603793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa7631-7c27-4d9f-8279-895d331f2a7b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:49.658813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 897/1807] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:49 2022] POST /identity/v3/users => generated 289 bytes in 606 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:49.663394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa7631-7c27-4d9f-8279-895d331f2a7b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:49.669776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-23fa7631-7c27-4d9f-8279-895d331f2a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:49.717448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa7631-7c27-4d9f-8279-895d331f2a7b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:49.721246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a455a-92d2-4b93-bfd9-c057329442e8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:49.728849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a4af4-63a7-4ce8-8463-fd7084444ee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:49.731321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a4af4-63a7-4ce8-8463-fd7084444ee2 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:49.731671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a4af4-63a7-4ce8-8463-fd7084444ee2 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:49.732022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a4af4-63a7-4ce8-8463-fd7084444ee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:49.737383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:49.742156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a4af4-63a7-4ce8-8463-fd7084444ee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:49.742661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:49.743532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:49.743976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a4af4-63a7-4ce8-8463-fd7084444ee2 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:49.745705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] PATH_INFO: `/v3/projects/cb0232fa025246b9a0f4fc1a8a6e328a` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:49.750286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb0232fa025246b9a0f4fc1a8a6e328a)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:49.757506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 911/1808] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:49.758422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a4af4-63a7-4ce8-8463-fd7084444ee2 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:49.759805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-be5a4af4-63a7-4ce8-8463-fd7084444ee2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:49.774469 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:49.832666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a4af4-63a7-4ce8-8463-fd7084444ee2 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:49.844087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749117ff-344a-42a2-81e8-169f41df078e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:49.845752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749117ff-344a-42a2-81e8-169f41df078e service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:49.846241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749117ff-344a-42a2-81e8-169f41df078e service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:49.846965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749117ff-344a-42a2-81e8-169f41df078e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:49.870637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749117ff-344a-42a2-81e8-169f41df078e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:49.888475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749117ff-344a-42a2-81e8-169f41df078e service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:49.901668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:49.926039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 912/1809] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:49.927324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749117ff-344a-42a2-81e8-169f41df078e service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:49.943551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:49.971203 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749117ff-344a-42a2-81e8-169f41df078e service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:49.972954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0232fa025246b9a0f4fc1a8a6e328a', 'request_id': 'req-65805689-4a61-41b2-aef3-44c3c8d86eaf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:49.998123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0232fa025246b9a0f4fc1a8a6e328a', 'request_id': 'req-65805689-4a61-41b2-aef3-44c3c8d86eaf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.026453 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0232fa025246b9a0f4fc1a8a6e328a', 'request_id': 'req-65805689-4a61-41b2-aef3-44c3c8d86eaf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.084419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] The token cache is being invalidate because project cb0232fa025246b9a0f4fc1a8a6e328a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:50.084959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.087310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.094710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 898/1810] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:49 2022] DELETE /identity/v3/projects/cb0232fa025246b9a0f4fc1a8a6e328a => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:50.102230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:50.107709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:50.209599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca726d75-bb2c-4389-93e6-d91519af1138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:50.210099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca726d75-bb2c-4389-93e6-d91519af1138 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:50.210347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca726d75-bb2c-4389-93e6-d91519af1138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:50.210625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca726d75-bb2c-4389-93e6-d91519af1138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:50.211363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca726d75-bb2c-4389-93e6-d91519af1138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:50.213020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca726d75-bb2c-4389-93e6-d91519af1138 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:50.234925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 913/1811] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:50.236023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca726d75-bb2c-4389-93e6-d91519af1138 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:50.238039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ca726d75-bb2c-4389-93e6-d91519af1138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:50.252439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca726d75-bb2c-4389-93e6-d91519af1138 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:50.257980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:50.259192 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:50.259626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:50.260043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] PATH_INFO: `/v3/users/e9f4d8190ae644ae88ac0cf252e850f6` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:50.261550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f4d8190ae644ae88ac0cf252e850f6)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:50.263780 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:50.332964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65805689-4a61-41b2-aef3-44c3c8d86eaf admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:50.347536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:50.348478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:50.348784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:50.349051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] PATH_INFO: `/v3/users/e37438f1264e4c4bbda467d962c66f62` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:50.356932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e37438f1264e4c4bbda467d962c66f62)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:50.359148 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:50.386755 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f4d8190ae644ae88ac0cf252e850f6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.405984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f4d8190ae644ae88ac0cf252e850f6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.406713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f4d8190ae644ae88ac0cf252e850f6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.407484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f4d8190ae644ae88ac0cf252e850f6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.431024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f4d8190ae644ae88ac0cf252e850f6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.460050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 914/1812] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:50 2022] DELETE /identity/v3/users/e9f4d8190ae644ae88ac0cf252e850f6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:50.461514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:50.464567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:50.512017 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e37438f1264e4c4bbda467d962c66f62'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.514303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e37438f1264e4c4bbda467d962c66f62'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.541186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e37438f1264e4c4bbda467d962c66f62'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.542103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e37438f1264e4c4bbda467d962c66f62'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.555230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e37438f1264e4c4bbda467d962c66f62'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.573003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1f91c-4e7b-4d2b-861b-0f53e6330d13 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:50.593660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:50.601815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:50.603527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:50.603982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] PATH_INFO: `/v3/users/4f70fa53427c4eafb06aef29f04d0b5e` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:50.611830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f70fa53427c4eafb06aef29f04d0b5e)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:50.619851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:50.623349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 899/1813] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:50 2022] DELETE /identity/v3/users/e37438f1264e4c4bbda467d962c66f62 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:50.624477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:50.630177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:50.681307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:50.795551 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f70fa53427c4eafb06aef29f04d0b5e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.813704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f70fa53427c4eafb06aef29f04d0b5e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.815529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f70fa53427c4eafb06aef29f04d0b5e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.816875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f70fa53427c4eafb06aef29f04d0b5e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.848793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-26cf15c0-84e0-47ad-b63e-21fa4b769823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f70fa53427c4eafb06aef29f04d0b5e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:50.866475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:50.881028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 915/1814] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:50 2022] DELETE /identity/v3/users/4f70fa53427c4eafb06aef29f04d0b5e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:50.882717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651236c-590d-4bf0-970d-13b2bbc2a3df None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:50.883237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651236c-590d-4bf0-970d-13b2bbc2a3df None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:50.883724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651236c-590d-4bf0-970d-13b2bbc2a3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:50.913976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee24f1-5001-4daa-bc79-158d669178e0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:50.920991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0651236c-590d-4bf0-970d-13b2bbc2a3df None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:50.922532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0651236c-590d-4bf0-970d-13b2bbc2a3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:51.107609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32289368-a144-4e02-8a6e-7792385342f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:51.108312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32289368-a144-4e02-8a6e-7792385342f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.108602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32289368-a144-4e02-8a6e-7792385342f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.108882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32289368-a144-4e02-8a6e-7792385342f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:51.109632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32289368-a144-4e02-8a6e-7792385342f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:51.127255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32289368-a144-4e02-8a6e-7792385342f2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:51.133461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 916/1815] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:51.136264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dff344-d25c-4621-ab7f-273d6d286824 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.140940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dff344-d25c-4621-ab7f-273d6d286824 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.140940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dff344-d25c-4621-ab7f-273d6d286824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:51.154023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 900/1816] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 530 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:51.158731 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32289368-a144-4e02-8a6e-7792385342f2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:51.162620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-32289368-a144-4e02-8a6e-7792385342f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:51.177127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-29dff344-d25c-4621-ab7f-273d6d286824 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:51.178554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-29dff344-d25c-4621-ab7f-273d6d286824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:51.187163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32289368-a144-4e02-8a6e-7792385342f2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:51.198650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 917/1817] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:51.200408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57d821-9dfd-4777-bc4c-3d7d0ffaffa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:51.201342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57d821-9dfd-4777-bc4c-3d7d0ffaffa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.201620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57d821-9dfd-4777-bc4c-3d7d0ffaffa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.201899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57d821-9dfd-4777-bc4c-3d7d0ffaffa1 admin admin] PATH_INFO: `/v3/projects/bcf80960d4884720afebaf1b613bdb03/users/09d63568f3914675841064cf5528138f/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:51.208940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dff344-d25c-4621-ab7f-273d6d286824 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:51.211074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-29dff344-d25c-4621-ab7f-273d6d286824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:51.239934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57d821-9dfd-4777-bc4c-3d7d0ffaffa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcf80960d4884720afebaf1b613bdb03, user_id=09d63568f3914675841064cf5528138f, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:51.242968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57d821-9dfd-4777-bc4c-3d7d0ffaffa1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:51.274330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 901/1818] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:51 2022] PUT /identity/v3/projects/bcf80960d4884720afebaf1b613bdb03/users/09d63568f3914675841064cf5528138f/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:51.274330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57d821-9dfd-4777-bc4c-3d7d0ffaffa1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:51.279690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-da57d821-9dfd-4777-bc4c-3d7d0ffaffa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:51.340121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dff344-d25c-4621-ab7f-273d6d286824 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:51.455811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:51.456321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57d821-9dfd-4777-bc4c-3d7d0ffaffa1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:51.468009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33daee8-99e3-42b1-827c-2da4072a7db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:51.468987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33daee8-99e3-42b1-827c-2da4072a7db7 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.469446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33daee8-99e3-42b1-827c-2da4072a7db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.469864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33daee8-99e3-42b1-827c-2da4072a7db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:51.503022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6ea48-bb8e-4dcc-b583-f05321b49051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:51.503727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6ea48-bb8e-4dcc-b583-f05321b49051 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.504013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6ea48-bb8e-4dcc-b583-f05321b49051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.504296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6ea48-bb8e-4dcc-b583-f05321b49051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:51.605676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33daee8-99e3-42b1-827c-2da4072a7db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:51.607491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33daee8-99e3-42b1-827c-2da4072a7db7 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:51.618220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 902/1819] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:04:51.619397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737e729-8ed3-49b7-af57-7a08b4eb5e41 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.619860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737e729-8ed3-49b7-af57-7a08b4eb5e41 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.620049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737e729-8ed3-49b7-af57-7a08b4eb5e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:51.651019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d737e729-8ed3-49b7-af57-7a08b4eb5e41 None None] MFA Rules not processed for user `09d63568f3914675841064cf5528138f`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:51.652453 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d737e729-8ed3-49b7-af57-7a08b4eb5e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:51.672245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:51.673019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6ea48-bb8e-4dcc-b583-f05321b49051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:51.676936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6ea48-bb8e-4dcc-b583-f05321b49051 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:51.703935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 918/1820] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:04:51.704898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6ea48-bb8e-4dcc-b583-f05321b49051 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:51.707914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-69b6ea48-bb8e-4dcc-b583-f05321b49051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:51.727174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6ea48-bb8e-4dcc-b583-f05321b49051 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:51.745572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:51.746494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.746775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.747053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] PATH_INFO: `/v3/projects/213d71ec77444306925e6afc1cad40b6` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:51.748407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213d71ec77444306925e6afc1cad40b6)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:51.751516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:51.793330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 903/1821] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:51.795270 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b959055-f06f-4cd2-a3fb-60e25a51710a None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.795729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b959055-f06f-4cd2-a3fb-60e25a51710a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.796198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b959055-f06f-4cd2-a3fb-60e25a51710a None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:51.798412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 904/1822] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 11:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:04:51.799896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b959055-f06f-4cd2-a3fb-60e25a51710a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:51.827761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b959055-f06f-4cd2-a3fb-60e25a51710a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:51.846870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213d71ec77444306925e6afc1cad40b6', 'request_id': 'req-49bda610-8c59-40d1-a734-4124d9f79992'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:51.860989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213d71ec77444306925e6afc1cad40b6', 'request_id': 'req-49bda610-8c59-40d1-a734-4124d9f79992'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:51.877155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213d71ec77444306925e6afc1cad40b6', 'request_id': 'req-49bda610-8c59-40d1-a734-4124d9f79992'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:51.901654 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] The token cache is being invalidate because project 213d71ec77444306925e6afc1cad40b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:51.902793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:51.903845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49bda610-8c59-40d1-a734-4124d9f79992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:51.909321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 919/1823] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:51 2022] DELETE /identity/v3/projects/213d71ec77444306925e6afc1cad40b6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:51.912207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9c8de-bd7b-4deb-8888-42302a4f6848 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.913018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9c8de-bd7b-4deb-8888-42302a4f6848 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.913470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9c8de-bd7b-4deb-8888-42302a4f6848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:51.928695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8921ea-d456-4c6e-9dcf-ed0335619309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:51.929753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8921ea-d456-4c6e-9dcf-ed0335619309 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.930215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8921ea-d456-4c6e-9dcf-ed0335619309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.930829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8921ea-d456-4c6e-9dcf-ed0335619309 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:51.932305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8921ea-d456-4c6e-9dcf-ed0335619309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:51.938010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-fde9c8de-bd7b-4deb-8888-42302a4f6848 None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:51.939398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fde9c8de-bd7b-4deb-8888-42302a4f6848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:51.946677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8921ea-d456-4c6e-9dcf-ed0335619309 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:51.993495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 905/1824] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:51 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:51.994939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a08c6-ab34-49a1-bd5f-8268f5066a32 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:51.995237 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a08c6-ab34-49a1-bd5f-8268f5066a32 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:51.995516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a08c6-ab34-49a1-bd5f-8268f5066a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.034022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-708a08c6-ab34-49a1-bd5f-8268f5066a32 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:52.036551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-708a08c6-ab34-49a1-bd5f-8268f5066a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.060630 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 920/1825] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:52.062764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9c8de-bd7b-4deb-8888-42302a4f6848 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:52.064882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fde9c8de-bd7b-4deb-8888-42302a4f6848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.077114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:52.088342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9c8de-bd7b-4deb-8888-42302a4f6848 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:52.148701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 906/1826] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:52.149501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a08c6-ab34-49a1-bd5f-8268f5066a32 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:52.151034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-708a08c6-ab34-49a1-bd5f-8268f5066a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.239139 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:52.291474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632ec09-646f-417e-876d-4db7464c923a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:52.292302 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632ec09-646f-417e-876d-4db7464c923a admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:52.292814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632ec09-646f-417e-876d-4db7464c923a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:52.292814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632ec09-646f-417e-876d-4db7464c923a admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.293503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632ec09-646f-417e-876d-4db7464c923a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:52.299688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632ec09-646f-417e-876d-4db7464c923a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:52.317875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a08c6-ab34-49a1-bd5f-8268f5066a32 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:52.326387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453f5b6-6141-472b-bf84-0708559210a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:52.327071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453f5b6-6141-472b-bf84-0708559210a7 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:52.327387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453f5b6-6141-472b-bf84-0708559210a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:52.327673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453f5b6-6141-472b-bf84-0708559210a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.339765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453f5b6-6141-472b-bf84-0708559210a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:52.341745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453f5b6-6141-472b-bf84-0708559210a7 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:52.351872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 921/1827] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:52 2022] POST /identity/v3/users => generated 290 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:52.360355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfdbe8-439a-415e-a61c-945e0556be88 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:52.360661 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfdbe8-439a-415e-a61c-945e0556be88 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:52.362201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 907/1828] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:52.362348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfdbe8-439a-415e-a61c-945e0556be88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.363895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453f5b6-6141-472b-bf84-0708559210a7 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:52.370788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c453f5b6-6141-472b-bf84-0708559210a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.395277 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453f5b6-6141-472b-bf84-0708559210a7 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:52.413468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:52.414563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:52.414865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:52.415044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] PATH_INFO: `/v3/projects/379971362a734eff897adfe04a7d2cdd` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.420086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] RBAC: Authorizing `identity:delete_project(project_id=379971362a734eff897adfe04a7d2cdd)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:52.422169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:52.442296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9fdfdbe8-439a-415e-a61c-945e0556be88 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:52.442296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9fdfdbe8-439a-415e-a61c-945e0556be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.452249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 922/1829] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:52.457694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfdbe8-439a-415e-a61c-945e0556be88 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:52.483130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfdbe8-439a-415e-a61c-945e0556be88 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:52.496443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3abac3-c512-4eea-8212-b626081d93c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:52.496443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3abac3-c512-4eea-8212-b626081d93c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:52.496443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3abac3-c512-4eea-8212-b626081d93c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:52.496443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3abac3-c512-4eea-8212-b626081d93c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.496970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3abac3-c512-4eea-8212-b626081d93c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:52.496970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3abac3-c512-4eea-8212-b626081d93c5 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:52.525687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 923/1830] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:52.526244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '379971362a734eff897adfe04a7d2cdd', 'request_id': 'req-3d34623c-a5f9-4df5-8928-d78906af840b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.526988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3abac3-c512-4eea-8212-b626081d93c5 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:52.529958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7b3abac3-c512-4eea-8212-b626081d93c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.538920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '379971362a734eff897adfe04a7d2cdd', 'request_id': 'req-3d34623c-a5f9-4df5-8928-d78906af840b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.547005 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '379971362a734eff897adfe04a7d2cdd', 'request_id': 'req-3d34623c-a5f9-4df5-8928-d78906af840b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.556575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3abac3-c512-4eea-8212-b626081d93c5 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:52.576028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] The token cache is being invalidate because project 379971362a734eff897adfe04a7d2cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:52.576511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.577827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.583608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 908/1831] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:52 2022] DELETE /identity/v3/projects/379971362a734eff897adfe04a7d2cdd => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:52.583608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:52.583608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.669352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:52.670675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:52.670971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:52.671421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] PATH_INFO: `/v3/projects/c01607c0fc0b4090a85320414d1def1f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.672752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c01607c0fc0b4090a85320414d1def1f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:52.676135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:52.697360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34623c-a5f9-4df5-8928-d78906af840b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:52.714515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dd3c6-39ce-4763-b4b5-d46b6bc67fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:52.716488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dd3c6-39ce-4763-b4b5-d46b6bc67fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:52.716770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dd3c6-39ce-4763-b4b5-d46b6bc67fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:52.717124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dd3c6-39ce-4763-b4b5-d46b6bc67fc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.717931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dd3c6-39ce-4763-b4b5-d46b6bc67fc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:52.718582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dd3c6-39ce-4763-b4b5-d46b6bc67fc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:52.721204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dd3c6-39ce-4763-b4b5-d46b6bc67fc4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:52.737665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 909/1832] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:52.738934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dd3c6-39ce-4763-b4b5-d46b6bc67fc4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:52.749493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ce5dd3c6-39ce-4763-b4b5-d46b6bc67fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.752421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c01607c0fc0b4090a85320414d1def1f', 'request_id': 'req-054e3184-1a35-41a1-90c7-6ce81fc8f842'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.762965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c01607c0fc0b4090a85320414d1def1f', 'request_id': 'req-054e3184-1a35-41a1-90c7-6ce81fc8f842'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.766635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dd3c6-39ce-4763-b4b5-d46b6bc67fc4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:52.770999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c01607c0fc0b4090a85320414d1def1f', 'request_id': 'req-054e3184-1a35-41a1-90c7-6ce81fc8f842'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.786367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] The token cache is being invalidate because project c01607c0fc0b4090a85320414d1def1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:52.786879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.787647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.792373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 924/1833] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:52 2022] DELETE /identity/v3/projects/c01607c0fc0b4090a85320414d1def1f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:52.793493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:52.795158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.801799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:52.817651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:52.845210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3948f46-c92d-493e-9c05-3e3c9fdad20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:52.846225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3948f46-c92d-493e-9c05-3e3c9fdad20a admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:52.846521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3948f46-c92d-493e-9c05-3e3c9fdad20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:52.846828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3948f46-c92d-493e-9c05-3e3c9fdad20a admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.847660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3948f46-c92d-493e-9c05-3e3c9fdad20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:52.849285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3948f46-c92d-493e-9c05-3e3c9fdad20a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:52.861774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 910/1834] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:52.863353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3948f46-c92d-493e-9c05-3e3c9fdad20a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:52.865182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c3948f46-c92d-493e-9c05-3e3c9fdad20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.877875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3948f46-c92d-493e-9c05-3e3c9fdad20a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:52.883507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d36974-b23b-4e43-97a8-1deeb91c6d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:52.884275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d36974-b23b-4e43-97a8-1deeb91c6d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:52.886903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d36974-b23b-4e43-97a8-1deeb91c6d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:52.886903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d36974-b23b-4e43-97a8-1deeb91c6d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.886903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d36974-b23b-4e43-97a8-1deeb91c6d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:52.887876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d36974-b23b-4e43-97a8-1deeb91c6d65 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:52.887984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e3184-1a35-41a1-90c7-6ce81fc8f842 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:52.894740 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:52.896122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:52.896577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:52.896997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] PATH_INFO: `/v3/users/708f229ff00a4f50bbb51faaeca65556` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:52.899138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=708f229ff00a4f50bbb51faaeca65556)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:52.903635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:52.930122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 911/1835] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:52 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:52.930122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d36974-b23b-4e43-97a8-1deeb91c6d65 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:52.932580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-28d36974-b23b-4e43-97a8-1deeb91c6d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:52.966349 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '708f229ff00a4f50bbb51faaeca65556'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.975120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '708f229ff00a4f50bbb51faaeca65556'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.975921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '708f229ff00a4f50bbb51faaeca65556'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.976545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '708f229ff00a4f50bbb51faaeca65556'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:52.989229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '708f229ff00a4f50bbb51faaeca65556'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:53.001561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d36974-b23b-4e43-97a8-1deeb91c6d65 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:53.006335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:53.007379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.007532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.007963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] PATH_INFO: `/v3/users/57a75ad0773e474281fb139fcb72e6af` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.009274 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=57a75ad0773e474281fb139fcb72e6af)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:53.011584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:53.014251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 925/1836] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:52 2022] DELETE /identity/v3/users/708f229ff00a4f50bbb51faaeca65556 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:53.015507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.017643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:53.085484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57a75ad0773e474281fb139fcb72e6af'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:53.086680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57a75ad0773e474281fb139fcb72e6af'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:53.105012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57a75ad0773e474281fb139fcb72e6af'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:53.106633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a68ad4-602d-459d-bf05-6197e1ad9a80 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:53.106813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57a75ad0773e474281fb139fcb72e6af'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:53.112052 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61778f4-2d5b-4ad6-935b-def1a41aab9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:53.119246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57a75ad0773e474281fb139fcb72e6af'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:53.126388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61778f4-2d5b-4ad6-935b-def1a41aab9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.127318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61778f4-2d5b-4ad6-935b-def1a41aab9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.128127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61778f4-2d5b-4ad6-935b-def1a41aab9a admin admin] PATH_INFO: `/v3/projects/d0e4e86274c84e40ba656ba040c1d913/users/1f568768ffe84647ac3f39336e186b1e/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.148174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 912/1837] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:52 2022] DELETE /identity/v3/users/57a75ad0773e474281fb139fcb72e6af => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:53.150122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.152106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:53.152745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61778f4-2d5b-4ad6-935b-def1a41aab9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e4e86274c84e40ba656ba040c1d913, user_id=1f568768ffe84647ac3f39336e186b1e, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:53.155650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61778f4-2d5b-4ad6-935b-def1a41aab9a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:53.176770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 926/1838] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:53 2022] PUT /identity/v3/projects/d0e4e86274c84e40ba656ba040c1d913/users/1f568768ffe84647ac3f39336e186b1e/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:53.189804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29f2d4-da21-4ec1-8e9c-7f20fe078dcf None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.190356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29f2d4-da21-4ec1-8e9c-7f20fe078dcf None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.191115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29f2d4-da21-4ec1-8e9c-7f20fe078dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.206571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:53.211453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:53.240710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-dc29f2d4-da21-4ec1-8e9c-7f20fe078dcf None None] MFA Rules not processed for user `1f568768ffe84647ac3f39336e186b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:53.241990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dc29f2d4-da21-4ec1-8e9c-7f20fe078dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:53.256664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21159af3-e74b-4317-9fce-e26ad12785bb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:53.349919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 927/1839] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:53.352097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae2d9c-f310-4359-8bca-0d091ba59ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:53.352898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae2d9c-f310-4359-8bca-0d091ba59ace admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.353078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29f2d4-da21-4ec1-8e9c-7f20fe078dcf None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.353246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae2d9c-f310-4359-8bca-0d091ba59ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.353534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae2d9c-f310-4359-8bca-0d091ba59ace admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.354589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae2d9c-f310-4359-8bca-0d091ba59ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:53.355123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dc29f2d4-da21-4ec1-8e9c-7f20fe078dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:53.356845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae2d9c-f310-4359-8bca-0d091ba59ace admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:53.371542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29f2d4-da21-4ec1-8e9c-7f20fe078dcf None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:53.378455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b81e0-de6b-4db1-be94-834c250c8740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:53.379210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b81e0-de6b-4db1-be94-834c250c8740 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.379671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b81e0-de6b-4db1-be94-834c250c8740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.380264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b81e0-de6b-4db1-be94-834c250c8740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.381179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b81e0-de6b-4db1-be94-834c250c8740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:53.383513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b81e0-de6b-4db1-be94-834c250c8740 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:53.390891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 913/1840] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:53 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:53.392616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae2d9c-f310-4359-8bca-0d091ba59ace admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.406840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae2d9c-f310-4359-8bca-0d091ba59ace admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:53.415394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee10e11-5b7c-4d82-b4a6-55bee7aa06fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:53.415661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee10e11-5b7c-4d82-b4a6-55bee7aa06fc admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.415937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee10e11-5b7c-4d82-b4a6-55bee7aa06fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.416267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee10e11-5b7c-4d82-b4a6-55bee7aa06fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.417047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee10e11-5b7c-4d82-b4a6-55bee7aa06fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:53.419072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee10e11-5b7c-4d82-b4a6-55bee7aa06fc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:53.428087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 928/1841] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:53 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:53.431518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 914/1842] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:53.433137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b81e0-de6b-4db1-be94-834c250c8740 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.434979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee10e11-5b7c-4d82-b4a6-55bee7aa06fc admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.450608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee10e11-5b7c-4d82-b4a6-55bee7aa06fc admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:53.527971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca30406-508c-4d2d-b08c-a02b1bfc1dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:53.529084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca30406-508c-4d2d-b08c-a02b1bfc1dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.529509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b81e0-de6b-4db1-be94-834c250c8740 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:53.529625 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca30406-508c-4d2d-b08c-a02b1bfc1dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.530463 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca30406-508c-4d2d-b08c-a02b1bfc1dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.531640 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca30406-508c-4d2d-b08c-a02b1bfc1dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:53.534351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca30406-508c-4d2d-b08c-a02b1bfc1dc1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:53.534492 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194793d2-5b96-4bc3-83e8-e2cde2a153b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:53.535175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194793d2-5b96-4bc3-83e8-e2cde2a153b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.535592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194793d2-5b96-4bc3-83e8-e2cde2a153b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.535881 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194793d2-5b96-4bc3-83e8-e2cde2a153b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.536702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194793d2-5b96-4bc3-83e8-e2cde2a153b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:53.540974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194793d2-5b96-4bc3-83e8-e2cde2a153b5 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:53.562622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 915/1843] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:04:53.567663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca30406-508c-4d2d-b08c-a02b1bfc1dc1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.596922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 929/1844] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:53 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:53.600820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194793d2-5b96-4bc3-83e8-e2cde2a153b5 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.604883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca30406-508c-4d2d-b08c-a02b1bfc1dc1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:53.615278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89be539-dd45-4566-96e6-0eec1a049480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:53.616677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194793d2-5b96-4bc3-83e8-e2cde2a153b5 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:53.616971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89be539-dd45-4566-96e6-0eec1a049480 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.617451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89be539-dd45-4566-96e6-0eec1a049480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.618389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89be539-dd45-4566-96e6-0eec1a049480 admin admin] PATH_INFO: `/v3/projects/86fba0c733be44b8beadefbe93da79c2/users/b43835cd6f224aeca7cb67e6c7ab31b5/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.627655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc37d4-e9cc-4c05-a714-51dd31e8d48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:53.628122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc37d4-e9cc-4c05-a714-51dd31e8d48d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.628610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc37d4-e9cc-4c05-a714-51dd31e8d48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.628998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc37d4-e9cc-4c05-a714-51dd31e8d48d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.630176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc37d4-e9cc-4c05-a714-51dd31e8d48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:53.633732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc37d4-e9cc-4c05-a714-51dd31e8d48d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:53.641200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89be539-dd45-4566-96e6-0eec1a049480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86fba0c733be44b8beadefbe93da79c2, user_id=b43835cd6f224aeca7cb67e6c7ab31b5, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:53.644168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89be539-dd45-4566-96e6-0eec1a049480 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:53.652860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 930/1845] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:53.654913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc37d4-e9cc-4c05-a714-51dd31e8d48d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.670535 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 916/1846] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:53 2022] PUT /identity/v3/projects/86fba0c733be44b8beadefbe93da79c2/users/b43835cd6f224aeca7cb67e6c7ab31b5/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:53.671179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc37d4-e9cc-4c05-a714-51dd31e8d48d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:53.683465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbdddc-d5f4-40cb-9cd4-078819fef2a5 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.683465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbdddc-d5f4-40cb-9cd4-078819fef2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.683465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbdddc-d5f4-40cb-9cd4-078819fef2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.729506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-93dbdddc-d5f4-40cb-9cd4-078819fef2a5 None None] MFA Rules not processed for user `b43835cd6f224aeca7cb67e6c7ab31b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:53.731583 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-93dbdddc-d5f4-40cb-9cd4-078819fef2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:53.741500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:53.791328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7254f-923a-4362-ae9c-be23be8b45f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:53.793630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:53.795266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7254f-923a-4362-ae9c-be23be8b45f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:53.795608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7254f-923a-4362-ae9c-be23be8b45f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:53.795902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7254f-923a-4362-ae9c-be23be8b45f6 admin admin] PATH_INFO: `/v3/projects/bd07684992fd44fab7fb9fe26f67f11c/users/891e740fc1404296b2d98bade0bf18e3/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:53.818596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7254f-923a-4362-ae9c-be23be8b45f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd07684992fd44fab7fb9fe26f67f11c, user_id=891e740fc1404296b2d98bade0bf18e3, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:53.823263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7254f-923a-4362-ae9c-be23be8b45f6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:53.855422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 931/1847] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:53 2022] PUT /identity/v3/projects/bd07684992fd44fab7fb9fe26f67f11c/users/891e740fc1404296b2d98bade0bf18e3/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:53.857108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7254f-923a-4362-ae9c-be23be8b45f6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.879680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 917/1848] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:53.891332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbdddc-d5f4-40cb-9cd4-078819fef2a5 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:53.892955 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-93dbdddc-d5f4-40cb-9cd4-078819fef2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:53.995684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbdddc-d5f4-40cb-9cd4-078819fef2a5 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:54.025618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7254f-923a-4362-ae9c-be23be8b45f6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:54.031062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a75b0d-fbfc-4ccf-99a3-9996f4b21586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:54.034991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a75b0d-fbfc-4ccf-99a3-9996f4b21586 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.035602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a75b0d-fbfc-4ccf-99a3-9996f4b21586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.036037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a75b0d-fbfc-4ccf-99a3-9996f4b21586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.037107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a75b0d-fbfc-4ccf-99a3-9996f4b21586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:54.039745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a75b0d-fbfc-4ccf-99a3-9996f4b21586 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:54.075946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 932/1849] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:54.079298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a75b0d-fbfc-4ccf-99a3-9996f4b21586 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:54.083608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f7a75b0d-fbfc-4ccf-99a3-9996f4b21586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:54.137472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4589546-397a-4398-9186-b6d3c847bc6d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bf4d9571d541bcbba87094198dc2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:54.138559 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a75b0d-fbfc-4ccf-99a3-9996f4b21586 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:54.140419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4589546-397a-4398-9186-b6d3c847bc6d service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.140937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4589546-397a-4398-9186-b6d3c847bc6d service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.141689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4589546-397a-4398-9186-b6d3c847bc6d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.163503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:54.164906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.165521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.168778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] PATH_INFO: `/v3/projects/6bcd924495314b99b99bb520a41202bb` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.171398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bcd924495314b99b99bb520a41202bb)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:54.177459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:54.284003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bcd924495314b99b99bb520a41202bb', 'request_id': 'req-9eac7d60-d4d5-4cc7-8fd7-e33982111212'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.296962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4589546-397a-4398-9186-b6d3c847bc6d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:54.298442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4589546-397a-4398-9186-b6d3c847bc6d service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:54.300708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bcd924495314b99b99bb520a41202bb', 'request_id': 'req-9eac7d60-d4d5-4cc7-8fd7-e33982111212'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.311790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bcd924495314b99b99bb520a41202bb', 'request_id': 'req-9eac7d60-d4d5-4cc7-8fd7-e33982111212'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.326261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 918/1850] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:54.326871 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] The token cache is being invalidate because project 6bcd924495314b99b99bb520a41202bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:54.327043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4589546-397a-4398-9186-b6d3c847bc6d service glance] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:54.327169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.328003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.330174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 933/1851] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:54 2022] DELETE /identity/v3/projects/6bcd924495314b99b99bb520a41202bb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:54.331237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:54.332697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:54.340993 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4589546-397a-4398-9186-b6d3c847bc6d service glance] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:54.398750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:54.446206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186aaaf8-f3d3-4e42-89cf-dd32fb475071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:54.447298 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aaaf8-f3d3-4e42-89cf-dd32fb475071 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.447671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aaaf8-f3d3-4e42-89cf-dd32fb475071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.448109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186aaaf8-f3d3-4e42-89cf-dd32fb475071 admin admin] PATH_INFO: `/v3/projects/bd07684992fd44fab7fb9fe26f67f11c/users/891e740fc1404296b2d98bade0bf18e3/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.450804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:54.451719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186aaaf8-f3d3-4e42-89cf-dd32fb475071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd07684992fd44fab7fb9fe26f67f11c, user_id=891e740fc1404296b2d98bade0bf18e3, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:54.479361 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186aaaf8-f3d3-4e42-89cf-dd32fb475071 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:54.481511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac7d60-d4d5-4cc7-8fd7-e33982111212 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:54.497782 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:54.499607 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.510617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 919/1852] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:54 2022] PUT /identity/v3/projects/bd07684992fd44fab7fb9fe26f67f11c/users/891e740fc1404296b2d98bade0bf18e3/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:54.510975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.510975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] PATH_INFO: `/v3/projects/60914d0ecf294ea79fbc4a7dfcc6db2b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.510975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=60914d0ecf294ea79fbc4a7dfcc6db2b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:54.510975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:54.511229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74d5cb-40c4-4395-acfe-d3e1a6be6f42 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.511229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74d5cb-40c4-4395-acfe-d3e1a6be6f42 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.511229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74d5cb-40c4-4395-acfe-d3e1a6be6f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.539089 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ba74d5cb-40c4-4395-acfe-d3e1a6be6f42 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:54.540342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ba74d5cb-40c4-4395-acfe-d3e1a6be6f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:54.585300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60914d0ecf294ea79fbc4a7dfcc6db2b', 'request_id': 'req-6078550c-018c-4477-adcd-f3d5d26c92ae'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.598717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60914d0ecf294ea79fbc4a7dfcc6db2b', 'request_id': 'req-6078550c-018c-4477-adcd-f3d5d26c92ae'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.605822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60914d0ecf294ea79fbc4a7dfcc6db2b', 'request_id': 'req-6078550c-018c-4477-adcd-f3d5d26c92ae'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.624425 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] The token cache is being invalidate because project 60914d0ecf294ea79fbc4a7dfcc6db2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:54.624831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.625636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.627671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 934/1853] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:54 2022] DELETE /identity/v3/projects/60914d0ecf294ea79fbc4a7dfcc6db2b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:54.631436 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:54.642742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 920/1854] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:54.643056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:54.648301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed03d9-3fe7-4c3d-a2d0-0ef8f3fdf331 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.648970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed03d9-3fe7-4c3d-a2d0-0ef8f3fdf331 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.649752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed03d9-3fe7-4c3d-a2d0-0ef8f3fdf331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.683111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-54ed03d9-3fe7-4c3d-a2d0-0ef8f3fdf331 None None] MFA Rules not processed for user `891e740fc1404296b2d98bade0bf18e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:54.687216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-54ed03d9-3fe7-4c3d-a2d0-0ef8f3fdf331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:54.734429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078550c-018c-4477-adcd-f3d5d26c92ae admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:54.740195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:54.741101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.741528 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.742076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] PATH_INFO: `/v3/users/c47aaccb33a64a9a99f06ad3d671c712` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.743471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c47aaccb33a64a9a99f06ad3d671c712)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:54.745451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:54.789506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 921/1855] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:54.790888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35767b8b-19fe-4157-a973-26a5fe33287c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.791189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35767b8b-19fe-4157-a973-26a5fe33287c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.791485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35767b8b-19fe-4157-a973-26a5fe33287c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.815776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-35767b8b-19fe-4157-a973-26a5fe33287c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:54.817648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-35767b8b-19fe-4157-a973-26a5fe33287c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:54.828197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 922/1856] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:54.829575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f409-2f59-48d9-b042-1513ffcce278 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.829783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f409-2f59-48d9-b042-1513ffcce278 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.830127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f409-2f59-48d9-b042-1513ffcce278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.845415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c47aaccb33a64a9a99f06ad3d671c712'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.853950 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-1a68f409-2f59-48d9-b042-1513ffcce278 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:54.855276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1a68f409-2f59-48d9-b042-1513ffcce278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:54.856572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c47aaccb33a64a9a99f06ad3d671c712'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.858406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c47aaccb33a64a9a99f06ad3d671c712'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.861489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c47aaccb33a64a9a99f06ad3d671c712'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.867804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 923/1857] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:54.867804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68f409-2f59-48d9-b042-1513ffcce278 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:54.869295 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1a68f409-2f59-48d9-b042-1513ffcce278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:54.883093 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-cd9d87bf-74af-4cde-8849-93029d69e97c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c47aaccb33a64a9a99f06ad3d671c712'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:54.900716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 935/1858] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:54 2022] DELETE /identity/v3/users/c47aaccb33a64a9a99f06ad3d671c712 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:54.902998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6da7d1-92e5-4fc8-acdf-a739a015dd11 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:54.903289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6da7d1-92e5-4fc8-acdf-a739a015dd11 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:54.904004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6da7d1-92e5-4fc8-acdf-a739a015dd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:54.916281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:54.925685 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:54.935258 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3b6da7d1-92e5-4fc8-acdf-a739a015dd11 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:54.936994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3b6da7d1-92e5-4fc8-acdf-a739a015dd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:54.945973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68f409-2f59-48d9-b042-1513ffcce278 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.018149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce1ea2-8279-44c2-9fae-8631b218f58c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.018300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 936/1859] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:55.019259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce1ea2-8279-44c2-9fae-8631b218f58c service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.019554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce1ea2-8279-44c2-9fae-8631b218f58c service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.019836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce1ea2-8279-44c2-9fae-8631b218f58c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.022796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6da7d1-92e5-4fc8-acdf-a739a015dd11 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.023417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-17ce1ea2-8279-44c2-9fae-8631b218f58c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:55.035466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6da7d1-92e5-4fc8-acdf-a739a015dd11 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.047044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b6c87-39fd-4b8a-8fd8-2cd3ae71a7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.048386 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b6c87-39fd-4b8a-8fd8-2cd3ae71a7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.048683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b6c87-39fd-4b8a-8fd8-2cd3ae71a7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.048963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b6c87-39fd-4b8a-8fd8-2cd3ae71a7a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.049705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b6c87-39fd-4b8a-8fd8-2cd3ae71a7a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:55.050400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b6c87-39fd-4b8a-8fd8-2cd3ae71a7a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.052053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b6c87-39fd-4b8a-8fd8-2cd3ae71a7a9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.068154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 937/1860] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:55.069488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79388a-edd1-41e1-91b0-b73cc1252d5c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.069777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79388a-edd1-41e1-91b0-b73cc1252d5c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.070456 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79388a-edd1-41e1-91b0-b73cc1252d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.089791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9d79388a-edd1-41e1-91b0-b73cc1252d5c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:55.091060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9d79388a-edd1-41e1-91b0-b73cc1252d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:55.097340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce1ea2-8279-44c2-9fae-8631b218f58c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.098956 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce1ea2-8279-44c2-9fae-8631b218f58c service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.100319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 938/1861] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:55.101200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79388a-edd1-41e1-91b0-b73cc1252d5c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.109901 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 924/1862] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:55.110718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce1ea2-8279-44c2-9fae-8631b218f58c service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.111857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79388a-edd1-41e1-91b0-b73cc1252d5c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.116843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd900c-d1f8-4656-8481-bb283bd2f3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.117590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd900c-d1f8-4656-8481-bb283bd2f3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.117947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd900c-d1f8-4656-8481-bb283bd2f3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.118370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd900c-d1f8-4656-8481-bb283bd2f3f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.119183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd900c-d1f8-4656-8481-bb283bd2f3f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:55.119652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd900c-d1f8-4656-8481-bb283bd2f3f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.121477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd900c-d1f8-4656-8481-bb283bd2f3f1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.122118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce1ea2-8279-44c2-9fae-8631b218f58c service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.128000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1758f5-288e-45bd-9571-81c9f8b5245a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.128736 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1758f5-288e-45bd-9571-81c9f8b5245a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.128992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1758f5-288e-45bd-9571-81c9f8b5245a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.129271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1758f5-288e-45bd-9571-81c9f8b5245a admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.130039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1758f5-288e-45bd-9571-81c9f8b5245a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.132830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1758f5-288e-45bd-9571-81c9f8b5245a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.139585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 939/1863] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:55.140615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd900c-d1f8-4656-8481-bb283bd2f3f1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.143993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bebd900c-d1f8-4656-8481-bb283bd2f3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:55.168998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 925/1864] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:55 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:55.170051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1758f5-288e-45bd-9571-81c9f8b5245a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.222171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd900c-d1f8-4656-8481-bb283bd2f3f1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.256754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1758f5-288e-45bd-9571-81c9f8b5245a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.262340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62d7cc-9937-4f80-826b-69f83fe1c7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.263114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62d7cc-9937-4f80-826b-69f83fe1c7fc admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.263395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62d7cc-9937-4f80-826b-69f83fe1c7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.263664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62d7cc-9937-4f80-826b-69f83fe1c7fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.264482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62d7cc-9937-4f80-826b-69f83fe1c7fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.266439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62d7cc-9937-4f80-826b-69f83fe1c7fc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.289080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bcb89-b5b6-4b1c-a038-1cbe6dd99371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.289839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bcb89-b5b6-4b1c-a038-1cbe6dd99371 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.290259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bcb89-b5b6-4b1c-a038-1cbe6dd99371 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.290584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bcb89-b5b6-4b1c-a038-1cbe6dd99371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.299549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 926/1865] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:55 2022] POST /identity/v3/projects => generated 407 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:55.300473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62d7cc-9937-4f80-826b-69f83fe1c7fc admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.301809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bcb89-b5b6-4b1c-a038-1cbe6dd99371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.303448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bcb89-b5b6-4b1c-a038-1cbe6dd99371 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.304198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1c62d7cc-9937-4f80-826b-69f83fe1c7fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:55.325768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 940/1866] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:55.326512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bcb89-b5b6-4b1c-a038-1cbe6dd99371 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.328978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:55.353380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bcb89-b5b6-4b1c-a038-1cbe6dd99371 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.417352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62d7cc-9937-4f80-826b-69f83fe1c7fc admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.417985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:55.447857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbffac-d3c4-49e4-b08c-9babd160db76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.449100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbffac-d3c4-49e4-b08c-9babd160db76 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.450261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbffac-d3c4-49e4-b08c-9babd160db76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.450261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbffac-d3c4-49e4-b08c-9babd160db76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.452421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-08dbffac-d3c4-49e4-b08c-9babd160db76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:55.473027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad63bea-8345-4a15-bcfe-082a0e8dc86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.473299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad63bea-8345-4a15-bcfe-082a0e8dc86b admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.473433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad63bea-8345-4a15-bcfe-082a0e8dc86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.473802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad63bea-8345-4a15-bcfe-082a0e8dc86b admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.474756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad63bea-8345-4a15-bcfe-082a0e8dc86b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.477001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad63bea-8345-4a15-bcfe-082a0e8dc86b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.506946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 941/1867] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:55 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:55.508608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad63bea-8345-4a15-bcfe-082a0e8dc86b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.522687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad63bea-8345-4a15-bcfe-082a0e8dc86b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.529089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9db28-2200-4319-8313-e46e81ddba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.529872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9db28-2200-4319-8313-e46e81ddba3f admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.530254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9db28-2200-4319-8313-e46e81ddba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.531025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9db28-2200-4319-8313-e46e81ddba3f admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.531960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9db28-2200-4319-8313-e46e81ddba3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.534660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9db28-2200-4319-8313-e46e81ddba3f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.560592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbffac-d3c4-49e4-b08c-9babd160db76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.563319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbffac-d3c4-49e4-b08c-9babd160db76 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.563890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 942/1868] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:55 2022] POST /identity/v3/users => generated 304 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:55.567875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9db28-2200-4319-8313-e46e81ddba3f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.577362 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 927/1869] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:55.578714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbffac-d3c4-49e4-b08c-9babd160db76 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.589584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9db28-2200-4319-8313-e46e81ddba3f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.590898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbffac-d3c4-49e4-b08c-9babd160db76 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.596964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b431035-2c7b-4eec-8fe3-65219562dc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.598377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b431035-2c7b-4eec-8fe3-65219562dc26 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.599131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b431035-2c7b-4eec-8fe3-65219562dc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.599373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c537-2b3d-4848-8877-c9de5457ab0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.599958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c537-2b3d-4848-8877-c9de5457ab0b admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.600194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b431035-2c7b-4eec-8fe3-65219562dc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.600292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c537-2b3d-4848-8877-c9de5457ab0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.600711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162c537-2b3d-4848-8877-c9de5457ab0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.601529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162c537-2b3d-4848-8877-c9de5457ab0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.602050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b431035-2c7b-4eec-8fe3-65219562dc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.603296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162c537-2b3d-4848-8877-c9de5457ab0b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.604690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b431035-2c7b-4eec-8fe3-65219562dc26 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.621234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 928/1870] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:55.625176 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c537-2b3d-4848-8877-c9de5457ab0b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.625480 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 943/1871] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:55.628358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b431035-2c7b-4eec-8fe3-65219562dc26 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.642475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b431035-2c7b-4eec-8fe3-65219562dc26 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.647446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162c537-2b3d-4848-8877-c9de5457ab0b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.654128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0ab9d-24d1-4db1-aa54-a23ab4a1ee5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.654377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777de1c5-8613-43e9-a81f-0ded73e63f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.654488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ab9d-24d1-4db1-aa54-a23ab4a1ee5b admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.654575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777de1c5-8613-43e9-a81f-0ded73e63f03 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.654645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ab9d-24d1-4db1-aa54-a23ab4a1ee5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.658806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777de1c5-8613-43e9-a81f-0ded73e63f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.659001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0ab9d-24d1-4db1-aa54-a23ab4a1ee5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.659079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777de1c5-8613-43e9-a81f-0ded73e63f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.659146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0ab9d-24d1-4db1-aa54-a23ab4a1ee5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.659146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0ab9d-24d1-4db1-aa54-a23ab4a1ee5b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.659270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777de1c5-8613-43e9-a81f-0ded73e63f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.662754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777de1c5-8613-43e9-a81f-0ded73e63f03 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.688889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 929/1872] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:55.691386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0ab9d-24d1-4db1-aa54-a23ab4a1ee5b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.693476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 944/1873] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:55.699149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777de1c5-8613-43e9-a81f-0ded73e63f03 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.714141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0ab9d-24d1-4db1-aa54-a23ab4a1ee5b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.714418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777de1c5-8613-43e9-a81f-0ded73e63f03 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:55.720829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ee127-26a1-4fb2-afdd-c45dbfdde139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.721665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee1ae7-5a55-42a1-a45a-e58c8da6f717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:55.721823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ee127-26a1-4fb2-afdd-c45dbfdde139 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.722036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ee127-26a1-4fb2-afdd-c45dbfdde139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.722407 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ee127-26a1-4fb2-afdd-c45dbfdde139 admin admin] PATH_INFO: `/v3/projects/d9e42229b1a5416c9412da025d51d293/users/0e87b5a6574b470bbff04f3949a9cafc/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.722725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee1ae7-5a55-42a1-a45a-e58c8da6f717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.723039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee1ae7-5a55-42a1-a45a-e58c8da6f717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.723318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee1ae7-5a55-42a1-a45a-e58c8da6f717 admin admin] PATH_INFO: `/v3/projects/9fead724efd5425691b837316a34b2d3/users/18882c697644475d8e67e1af9c6ddfa5/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.743051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee1ae7-5a55-42a1-a45a-e58c8da6f717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fead724efd5425691b837316a34b2d3, user_id=18882c697644475d8e67e1af9c6ddfa5, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.743652 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ee127-26a1-4fb2-afdd-c45dbfdde139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e42229b1a5416c9412da025d51d293, user_id=0e87b5a6574b470bbff04f3949a9cafc, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:55.747261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee1ae7-5a55-42a1-a45a-e58c8da6f717 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.748134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ee127-26a1-4fb2-afdd-c45dbfdde139 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:55.768210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 945/1874] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:55 2022] PUT /identity/v3/projects/9fead724efd5425691b837316a34b2d3/users/18882c697644475d8e67e1af9c6ddfa5/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:55.773358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 930/1875] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:55 2022] PUT /identity/v3/projects/d9e42229b1a5416c9412da025d51d293/users/0e87b5a6574b470bbff04f3949a9cafc/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:55.787488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7d23a-1cfe-41bf-94d3-c5e27d23794c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.787775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7d23a-1cfe-41bf-94d3-c5e27d23794c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.788019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7d23a-1cfe-41bf-94d3-c5e27d23794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.788388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc56882-0ea4-4eb7-b728-bfdbafda1da0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:55.789151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc56882-0ea4-4eb7-b728-bfdbafda1da0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:55.789501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc56882-0ea4-4eb7-b728-bfdbafda1da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:55.824047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5cc56882-0ea4-4eb7-b728-bfdbafda1da0 None None] MFA Rules not processed for user `18882c697644475d8e67e1af9c6ddfa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:55.824820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5cc56882-0ea4-4eb7-b728-bfdbafda1da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:55.825590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e9a7d23a-1cfe-41bf-94d3-c5e27d23794c None None] MFA Rules not processed for user `0e87b5a6574b470bbff04f3949a9cafc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:55.826909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e9a7d23a-1cfe-41bf-94d3-c5e27d23794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:55.956673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 931/1876] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:55 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:55.958063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7d23a-1cfe-41bf-94d3-c5e27d23794c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:55.963185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e9a7d23a-1cfe-41bf-94d3-c5e27d23794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:55.971287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:56.001266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 946/1877] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:56.002308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc56882-0ea4-4eb7-b728-bfdbafda1da0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.096467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:56.114211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc56882-0ea4-4eb7-b728-bfdbafda1da0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.115518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7d23a-1cfe-41bf-94d3-c5e27d23794c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.121831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7f98b-79a5-4bba-9a14-84adf23bd8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.121831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7f98b-79a5-4bba-9a14-84adf23bd8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.121831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7f98b-79a5-4bba-9a14-84adf23bd8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.122324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7f98b-79a5-4bba-9a14-84adf23bd8e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.123363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7f98b-79a5-4bba-9a14-84adf23bd8e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.125884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7f98b-79a5-4bba-9a14-84adf23bd8e2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.141695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.143081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.148035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.148530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] PATH_INFO: `/v3/projects/032ec5272572486c98dd352dd9635638` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.150624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=032ec5272572486c98dd352dd9635638)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.152886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.174579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 947/1878] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:56 2022] POST /identity/v3/projects => generated 409 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:56.175880 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7f98b-79a5-4bba-9a14-84adf23bd8e2 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.284809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032ec5272572486c98dd352dd9635638', 'request_id': 'req-47e7d4f7-4106-4794-8f30-5478a0152222'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:56.296559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032ec5272572486c98dd352dd9635638', 'request_id': 'req-47e7d4f7-4106-4794-8f30-5478a0152222'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:56.305574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032ec5272572486c98dd352dd9635638', 'request_id': 'req-47e7d4f7-4106-4794-8f30-5478a0152222'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:56.306212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7f98b-79a5-4bba-9a14-84adf23bd8e2 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.330506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] The token cache is being invalidate because project 032ec5272572486c98dd352dd9635638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:56.330718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:56.331436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:56.333347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 932/1879] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:55 2022] DELETE /identity/v3/projects/032ec5272572486c98dd352dd9635638 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:56.337458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.339442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:56.382364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dc80a-5a25-461f-9135-afe02b4c4e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.383163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dc80a-5a25-461f-9135-afe02b4c4e75 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.383442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dc80a-5a25-461f-9135-afe02b4c4e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.383715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dc80a-5a25-461f-9135-afe02b4c4e75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.384486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dc80a-5a25-461f-9135-afe02b4c4e75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.386776 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dc80a-5a25-461f-9135-afe02b4c4e75 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.416412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7d4f7-4106-4794-8f30-5478a0152222 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.425797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 948/1880] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:56 2022] POST /identity/v3/projects => generated 379 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:56.425797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dc80a-5a25-461f-9135-afe02b4c4e75 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.431550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8c0dc80a-5a25-461f-9135-afe02b4c4e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:56.434592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b9f22-9e0f-4f3b-85e0-9e8e01a8963a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.435470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b9f22-9e0f-4f3b-85e0-9e8e01a8963a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.435760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b9f22-9e0f-4f3b-85e0-9e8e01a8963a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.436007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b9f22-9e0f-4f3b-85e0-9e8e01a8963a admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.436826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b9f22-9e0f-4f3b-85e0-9e8e01a8963a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.438913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b9f22-9e0f-4f3b-85e0-9e8e01a8963a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.466417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 933/1881] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:56 2022] POST /identity/v3/users => generated 311 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:56.467575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b9f22-9e0f-4f3b-85e0-9e8e01a8963a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.469998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-391b9f22-9e0f-4f3b-85e0-9e8e01a8963a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:56.485222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b9f22-9e0f-4f3b-85e0-9e8e01a8963a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.518608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:56.526726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dc80a-5a25-461f-9135-afe02b4c4e75 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.531741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.532625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.532907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.533244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] PATH_INFO: `/v3/users/0f51ac1db66f48ec97770d8e7ad2737a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.534511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f51ac1db66f48ec97770d8e7ad2737a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.536448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.543374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:56.562015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448703-5a68-44dc-9eac-bb9a6e05e676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.562966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448703-5a68-44dc-9eac-bb9a6e05e676 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.563318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448703-5a68-44dc-9eac-bb9a6e05e676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.563664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448703-5a68-44dc-9eac-bb9a6e05e676 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.564564 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08448703-5a68-44dc-9eac-bb9a6e05e676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.566433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08448703-5a68-44dc-9eac-bb9a6e05e676 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.598737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 934/1882] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:56 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:56.599972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448703-5a68-44dc-9eac-bb9a6e05e676 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.601845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f51ac1db66f48ec97770d8e7ad2737a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:56.614732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448703-5a68-44dc-9eac-bb9a6e05e676 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.615139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f51ac1db66f48ec97770d8e7ad2737a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:56.616139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f51ac1db66f48ec97770d8e7ad2737a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:56.616946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f51ac1db66f48ec97770d8e7ad2737a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:56.622418 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935098d1-4b7b-4422-8cda-77262a0cb6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.623842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935098d1-4b7b-4422-8cda-77262a0cb6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.624203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935098d1-4b7b-4422-8cda-77262a0cb6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.624578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935098d1-4b7b-4422-8cda-77262a0cb6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.625399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935098d1-4b7b-4422-8cda-77262a0cb6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.628088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935098d1-4b7b-4422-8cda-77262a0cb6e0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.643363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 935/1883] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:56.644789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935098d1-4b7b-4422-8cda-77262a0cb6e0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.647186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f51ac1db66f48ec97770d8e7ad2737a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:56.647838 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-935098d1-4b7b-4422-8cda-77262a0cb6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:56.661728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 949/1884] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:56 2022] DELETE /identity/v3/users/0f51ac1db66f48ec97770d8e7ad2737a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:56.662853 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.664232 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:56.665928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935098d1-4b7b-4422-8cda-77262a0cb6e0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.744136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf4ded-c094-43e7-8864-9866ad133210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.744911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf4ded-c094-43e7-8864-9866ad133210 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.745264 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf4ded-c094-43e7-8864-9866ad133210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.745680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf4ded-c094-43e7-8864-9866ad133210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.746054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78070d9-5280-4c4d-b76b-9a4085de2f74 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.746691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf4ded-c094-43e7-8864-9866ad133210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.749070 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf4ded-c094-43e7-8864-9866ad133210 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.751679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210b6ef-46ec-4df4-9835-8707d451dd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.752952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210b6ef-46ec-4df4-9835-8707d451dd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.753333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210b6ef-46ec-4df4-9835-8707d451dd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.753719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210b6ef-46ec-4df4-9835-8707d451dd3c admin admin] PATH_INFO: `/v3/projects/07177cd28e714cd99910e60c2806f994/users/62756609670b484383b8427321a65bbc/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.761826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 936/1885] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:56.765508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf4ded-c094-43e7-8864-9866ad133210 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.772036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210b6ef-46ec-4df4-9835-8707d451dd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=07177cd28e714cd99910e60c2806f994, user_id=62756609670b484383b8427321a65bbc, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.780938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf4ded-c094-43e7-8864-9866ad133210 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.782176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210b6ef-46ec-4df4-9835-8707d451dd3c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.787202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0806f0d-7354-4a57-ac55-9bbe36fcc877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.788359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0806f0d-7354-4a57-ac55-9bbe36fcc877 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.788713 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0806f0d-7354-4a57-ac55-9bbe36fcc877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.789461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0806f0d-7354-4a57-ac55-9bbe36fcc877 admin admin] PATH_INFO: `/v3/projects/15535ad3ba8149bdb828c30d159c1554/users/64b183ac0f324a2fafa321eee79744e7/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.805409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 950/1886] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:56 2022] PUT /identity/v3/projects/07177cd28e714cd99910e60c2806f994/users/62756609670b484383b8427321a65bbc/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:56.806767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210b6ef-46ec-4df4-9835-8707d451dd3c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.816900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0806f0d-7354-4a57-ac55-9bbe36fcc877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15535ad3ba8149bdb828c30d159c1554, user_id=64b183ac0f324a2fafa321eee79744e7, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.820657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0806f0d-7354-4a57-ac55-9bbe36fcc877 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.840745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 937/1887] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:56 2022] PUT /identity/v3/projects/15535ad3ba8149bdb828c30d159c1554/users/64b183ac0f324a2fafa321eee79744e7/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:04:56.844085 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0806f0d-7354-4a57-ac55-9bbe36fcc877 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.894938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210b6ef-46ec-4df4-9835-8707d451dd3c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.916672 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0806f0d-7354-4a57-ac55-9bbe36fcc877 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.917729 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:56.924331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e616ea-c093-4e22-b26c-af72df16ec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.925170 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e616ea-c093-4e22-b26c-af72df16ec9e admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.926489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e616ea-c093-4e22-b26c-af72df16ec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.926598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e616ea-c093-4e22-b26c-af72df16ec9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.927649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e616ea-c093-4e22-b26c-af72df16ec9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.929461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e616ea-c093-4e22-b26c-af72df16ec9e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.950133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 938/1888] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:04:56.956082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e616ea-c093-4e22-b26c-af72df16ec9e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:56.969129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e616ea-c093-4e22-b26c-af72df16ec9e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:56.974618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e0a55-6470-4c5a-8bee-a72d47cd9ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.975815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e0a55-6470-4c5a-8bee-a72d47cd9ece admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.976148 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e0a55-6470-4c5a-8bee-a72d47cd9ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.976871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e0a55-6470-4c5a-8bee-a72d47cd9ece admin admin] PATH_INFO: `/v3/projects/15535ad3ba8149bdb828c30d159c1554/users/64b183ac0f324a2fafa321eee79744e7/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.978675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e0a55-6470-4c5a-8bee-a72d47cd9ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=15535ad3ba8149bdb828c30d159c1554, user_id=64b183ac0f324a2fafa321eee79744e7, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.981567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e0a55-6470-4c5a-8bee-a72d47cd9ece admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.982535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943de1b8-1f5a-457e-8c02-91d8924a5a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:56.983218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de1b8-1f5a-457e-8c02-91d8924a5a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:56.983494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de1b8-1f5a-457e-8c02-91d8924a5a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:56.984462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de1b8-1f5a-457e-8c02-91d8924a5a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:56.984462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943de1b8-1f5a-457e-8c02-91d8924a5a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:56.986282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943de1b8-1f5a-457e-8c02-91d8924a5a2a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:56.988157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:56.994677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 939/1889] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:56 2022] PUT /identity/v3/projects/15535ad3ba8149bdb828c30d159c1554/users/64b183ac0f324a2fafa321eee79744e7/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:04:56.999702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 951/1890] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:57.010120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e0a55-6470-4c5a-8bee-a72d47cd9ece admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:57.018699 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e3cf4-56cc-43af-ba72-c7e45f34921c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:57.020026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e3cf4-56cc-43af-ba72-c7e45f34921c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:57.020486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e3cf4-56cc-43af-ba72-c7e45f34921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:57.052243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d67e3cf4-56cc-43af-ba72-c7e45f34921c None None] MFA Rules not processed for user `64b183ac0f324a2fafa321eee79744e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:57.053626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d67e3cf4-56cc-43af-ba72-c7e45f34921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:57.088793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e0a55-6470-4c5a-8bee-a72d47cd9ece admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:57.093952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e8be2-d6da-453f-a7a3-6e2512d6cacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:57.094883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8be2-d6da-453f-a7a3-6e2512d6cacf admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:57.095159 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8be2-d6da-453f-a7a3-6e2512d6cacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:57.095433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e8be2-d6da-453f-a7a3-6e2512d6cacf admin admin] PATH_INFO: `/v3/projects/07177cd28e714cd99910e60c2806f994/users/62756609670b484383b8427321a65bbc/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:57.097413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e8be2-d6da-453f-a7a3-6e2512d6cacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=07177cd28e714cd99910e60c2806f994, user_id=62756609670b484383b8427321a65bbc, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:57.100334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e8be2-d6da-453f-a7a3-6e2512d6cacf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:57.112325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 940/1891] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:57 2022] PUT /identity/v3/projects/07177cd28e714cd99910e60c2806f994/users/62756609670b484383b8427321a65bbc/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:04:57.123238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15193514-2ac2-46f2-8151-83f854994c0a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:57.123577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15193514-2ac2-46f2-8151-83f854994c0a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:57.124862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15193514-2ac2-46f2-8151-83f854994c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:57.157966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 952/1892] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:57.165760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-15193514-2ac2-46f2-8151-83f854994c0a None None] MFA Rules not processed for user `62756609670b484383b8427321a65bbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:57.167838 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-15193514-2ac2-46f2-8151-83f854994c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:57.169836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e3cf4-56cc-43af-ba72-c7e45f34921c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:57.171689 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d67e3cf4-56cc-43af-ba72-c7e45f34921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:57.265562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e3cf4-56cc-43af-ba72-c7e45f34921c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:57.306167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 941/1893] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:57 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:57.355432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15193514-2ac2-46f2-8151-83f854994c0a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:57.356895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-15193514-2ac2-46f2-8151-83f854994c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:57.365725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f28c3c-3d53-4d74-91fc-29a834531945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:57.366487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f28c3c-3d53-4d74-91fc-29a834531945 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:57.366767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f28c3c-3d53-4d74-91fc-29a834531945 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:57.367042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f28c3c-3d53-4d74-91fc-29a834531945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:57.374958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:57.483914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f28c3c-3d53-4d74-91fc-29a834531945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:57.485575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f28c3c-3d53-4d74-91fc-29a834531945 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:57.492299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15193514-2ac2-46f2-8151-83f854994c0a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:57.505403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c4265-897f-4b13-be79-7ac2045395da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:57.506101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c4265-897f-4b13-be79-7ac2045395da service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:57.506386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c4265-897f-4b13-be79-7ac2045395da service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:57.506662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c4265-897f-4b13-be79-7ac2045395da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:57.510938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 953/1894] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:04:57.531915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c4265-897f-4b13-be79-7ac2045395da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:57.533590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c4265-897f-4b13-be79-7ac2045395da service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:57.578848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 942/1895] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:04:57.735889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f28c3c-3d53-4d74-91fc-29a834531945 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:57.737038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-49f28c3c-3d53-4d74-91fc-29a834531945 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:57.908251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f28c3c-3d53-4d74-91fc-29a834531945 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:57.917737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:57.919199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:57.919588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:57.919955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] PATH_INFO: `/v3/projects/2d38a45c88e748dd92091220957cc75d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:57.921813 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d38a45c88e748dd92091220957cc75d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:57.923494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c4265-897f-4b13-be79-7ac2045395da service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:57.936545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:57.943289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c4265-897f-4b13-be79-7ac2045395da service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:57.953586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf2ee8-a5f9-4f58-98b1-7fbddf74f917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:57.954795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf2ee8-a5f9-4f58-98b1-7fbddf74f917 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:57.955078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf2ee8-a5f9-4f58-98b1-7fbddf74f917 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:57.955352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf2ee8-a5f9-4f58-98b1-7fbddf74f917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:57.957599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-40cf2ee8-a5f9-4f58-98b1-7fbddf74f917 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:57.997464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:58.045571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d38a45c88e748dd92091220957cc75d', 'request_id': 'req-5f0a47dd-b768-40ef-86d6-388f34b4901b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:58.060903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d38a45c88e748dd92091220957cc75d', 'request_id': 'req-5f0a47dd-b768-40ef-86d6-388f34b4901b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:58.068042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d38a45c88e748dd92091220957cc75d', 'request_id': 'req-5f0a47dd-b768-40ef-86d6-388f34b4901b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:58.081900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] The token cache is being invalidate because project 2d38a45c88e748dd92091220957cc75d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:58.082328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:58.082912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf2ee8-a5f9-4f58-98b1-7fbddf74f917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:58.083186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5f0a47dd-b768-40ef-86d6-388f34b4901b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:58.088120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf2ee8-a5f9-4f58-98b1-7fbddf74f917 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:58.088553 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 954/1896] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:57 2022] DELETE /identity/v3/projects/2d38a45c88e748dd92091220957cc75d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:58.091694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9303e8-565a-4196-a604-3e4cb44c1176 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.091988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9303e8-565a-4196-a604-3e4cb44c1176 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.092269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9303e8-565a-4196-a604-3e4cb44c1176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.097297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 943/1897] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:04:58.117724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ae9303e8-565a-4196-a604-3e4cb44c1176 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:58.119073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ae9303e8-565a-4196-a604-3e4cb44c1176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:58.158576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:58.218486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 955/1898] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:04:58.227164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc7614-0239-4933-a8be-25bb214a71bf None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.227770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc7614-0239-4933-a8be-25bb214a71bf None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.228225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc7614-0239-4933-a8be-25bb214a71bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.253695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4bdc7614-0239-4933-a8be-25bb214a71bf None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:58.257525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4bdc7614-0239-4933-a8be-25bb214a71bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:58.269090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 944/1899] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:58.273760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9303e8-565a-4196-a604-3e4cb44c1176 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:58.284349 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9303e8-565a-4196-a604-3e4cb44c1176 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:58.291718 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589603ae-490f-47a1-9a44-0ff508ab99ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:58.292581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589603ae-490f-47a1-9a44-0ff508ab99ad admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.292928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589603ae-490f-47a1-9a44-0ff508ab99ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.293210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589603ae-490f-47a1-9a44-0ff508ab99ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.293955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589603ae-490f-47a1-9a44-0ff508ab99ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:04:58.294303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589603ae-490f-47a1-9a44-0ff508ab99ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:58.295990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589603ae-490f-47a1-9a44-0ff508ab99ad admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:58.313678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 956/1900] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:04:58.319339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc7614-0239-4933-a8be-25bb214a71bf None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:58.336570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc7614-0239-4933-a8be-25bb214a71bf None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:58.349214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114997f-20b8-4e93-8e0e-9b42bbeaeb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:58.350132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114997f-20b8-4e93-8e0e-9b42bbeaeb5c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.350263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114997f-20b8-4e93-8e0e-9b42bbeaeb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.350469 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114997f-20b8-4e93-8e0e-9b42bbeaeb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.351207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114997f-20b8-4e93-8e0e-9b42bbeaeb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:58.357084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114997f-20b8-4e93-8e0e-9b42bbeaeb5c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:58.443986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 945/1901] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:58 2022] POST /identity/v3/projects => generated 363 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:58.449268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589603ae-490f-47a1-9a44-0ff508ab99ad admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:58.465735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589603ae-490f-47a1-9a44-0ff508ab99ad admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:58.566505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4fdb3-4096-4d73-9e92-9d11e269c110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:58.568590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4fdb3-4096-4d73-9e92-9d11e269c110 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.569043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4fdb3-4096-4d73-9e92-9d11e269c110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.569578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4fdb3-4096-4d73-9e92-9d11e269c110 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.571307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4fdb3-4096-4d73-9e92-9d11e269c110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:58.574981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4fdb3-4096-4d73-9e92-9d11e269c110 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:58.616420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 957/1902] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:58 2022] POST /identity/v3/users => generated 282 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:04:58.617817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114997f-20b8-4e93-8e0e-9b42bbeaeb5c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:58.631787 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114997f-20b8-4e93-8e0e-9b42bbeaeb5c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:58.644371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b24a4-b1b1-41a0-ae32-1af37c82d8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:58.645769 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b24a4-b1b1-41a0-ae32-1af37c82d8cd admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.646080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b24a4-b1b1-41a0-ae32-1af37c82d8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.646298 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b24a4-b1b1-41a0-ae32-1af37c82d8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.647093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b24a4-b1b1-41a0-ae32-1af37c82d8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:58.649214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b24a4-b1b1-41a0-ae32-1af37c82d8cd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:58.665145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 946/1903] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:58.669338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4fdb3-4096-4d73-9e92-9d11e269c110 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:58.682665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4fdb3-4096-4d73-9e92-9d11e269c110 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:58.690517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0add0-d8e0-42fa-be60-2f3e6129435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:58.691274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0add0-d8e0-42fa-be60-2f3e6129435f admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.691609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0add0-d8e0-42fa-be60-2f3e6129435f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.691896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0add0-d8e0-42fa-be60-2f3e6129435f admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.692645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0add0-d8e0-42fa-be60-2f3e6129435f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:58.694433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0add0-d8e0-42fa-be60-2f3e6129435f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:58.718519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 958/1904] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:58.718839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b24a4-b1b1-41a0-ae32-1af37c82d8cd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:58.719829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0add0-d8e0-42fa-be60-2f3e6129435f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:58.722893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ec8b24a4-b1b1-41a0-ae32-1af37c82d8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:58.734564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0add0-d8e0-42fa-be60-2f3e6129435f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:58.738543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b24a4-b1b1-41a0-ae32-1af37c82d8cd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:58.739682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1bc60-851d-42cc-8c4b-5c17ee78591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:58.740596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1bc60-851d-42cc-8c4b-5c17ee78591a admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.740880 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1bc60-851d-42cc-8c4b-5c17ee78591a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.741163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1bc60-851d-42cc-8c4b-5c17ee78591a admin admin] PATH_INFO: `/v3/projects/569e28e736f440fc9337bf843707f261/users/6b3be1bc8c8c42faae2b5bac34646610/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.743519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:58.745122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.745451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.745740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] PATH_INFO: `/v3/users/18882c697644475d8e67e1af9c6ddfa5` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.747026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18882c697644475d8e67e1af9c6ddfa5)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:58.749049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:58.759177 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1bc60-851d-42cc-8c4b-5c17ee78591a admin admin] RBAC: Authorizing `identity:create_grant(project_id=569e28e736f440fc9337bf843707f261, user_id=6b3be1bc8c8c42faae2b5bac34646610, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:58.762184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1bc60-851d-42cc-8c4b-5c17ee78591a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:58.783794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 959/1905] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:58 2022] PUT /identity/v3/projects/569e28e736f440fc9337bf843707f261/users/6b3be1bc8c8c42faae2b5bac34646610/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:58.795767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf689-1717-4220-ba33-c0f03add4e05 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.796063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf689-1717-4220-ba33-c0f03add4e05 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.796370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf689-1717-4220-ba33-c0f03add4e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.815982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18882c697644475d8e67e1af9c6ddfa5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:58.816641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18882c697644475d8e67e1af9c6ddfa5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:58.825244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-07ccf689-1717-4220-ba33-c0f03add4e05 None None] MFA Rules not processed for user `6b3be1bc8c8c42faae2b5bac34646610`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:58.825773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18882c697644475d8e67e1af9c6ddfa5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:58.826449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18882c697644475d8e67e1af9c6ddfa5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:58.826864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-07ccf689-1717-4220-ba33-c0f03add4e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:58.833856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-28743d2e-a91b-45ee-a643-6d2dc84ea6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18882c697644475d8e67e1af9c6ddfa5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:58.852444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 947/1906] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:58 2022] DELETE /identity/v3/users/18882c697644475d8e67e1af9c6ddfa5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:04:58.858767 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61db30c-2236-4831-a2cb-937e985dc4c4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:58.859052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61db30c-2236-4831-a2cb-937e985dc4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:58.859335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61db30c-2236-4831-a2cb-937e985dc4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:58.883196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e61db30c-2236-4831-a2cb-937e985dc4c4 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:58.884681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e61db30c-2236-4831-a2cb-937e985dc4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:58.907286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 960/1907] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:58 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:58.908347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccf689-1717-4220-ba33-c0f03add4e05 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:58.910001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-07ccf689-1717-4220-ba33-c0f03add4e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:58.915682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:58.989106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccf689-1717-4220-ba33-c0f03add4e05 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:59.059877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3131f-0f0c-4778-adc8-ab05305532ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:59.061013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3131f-0f0c-4778-adc8-ab05305532ac admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:59.061013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3131f-0f0c-4778-adc8-ab05305532ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:59.061159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3131f-0f0c-4778-adc8-ab05305532ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:59.061793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3131f-0f0c-4778-adc8-ab05305532ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:59.063969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3131f-0f0c-4778-adc8-ab05305532ac admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:59.064374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:59.087789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 948/1908] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:04:59.099298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 961/1909] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:04:58 2022] POST /identity/v3/projects => generated 363 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:04:59.101013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61db30c-2236-4831-a2cb-937e985dc4c4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:59.102644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e61db30c-2236-4831-a2cb-937e985dc4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:59.103251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3131f-0f0c-4778-adc8-ab05305532ac admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:59.182988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61db30c-2236-4831-a2cb-937e985dc4c4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:59.190676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3131f-0f0c-4778-adc8-ab05305532ac admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:59.196275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f3145-a52f-4567-98d2-04cfaed0c5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:59.196969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f3145-a52f-4567-98d2-04cfaed0c5ab admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:59.197248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f3145-a52f-4567-98d2-04cfaed0c5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:59.197760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f3145-a52f-4567-98d2-04cfaed0c5ab admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:59.198677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f3145-a52f-4567-98d2-04cfaed0c5ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:59.201221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f3145-a52f-4567-98d2-04cfaed0c5ab admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:59.239675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 962/1910] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:04:59 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:04:59.243862 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f3145-a52f-4567-98d2-04cfaed0c5ab admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:59.247606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f847b-131a-4805-b43f-53026faa621d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:59.248349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f847b-131a-4805-b43f-53026faa621d service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:59.248691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f847b-131a-4805-b43f-53026faa621d service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:59.249025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f847b-131a-4805-b43f-53026faa621d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:59.255101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f3145-a52f-4567-98d2-04cfaed0c5ab admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:59.258995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f847b-131a-4805-b43f-53026faa621d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:59.260200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bbf11-1bd2-410b-a4d4-30c29444f560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:59.260873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f847b-131a-4805-b43f-53026faa621d service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:59.261642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bbf11-1bd2-410b-a4d4-30c29444f560 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:59.261926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bbf11-1bd2-410b-a4d4-30c29444f560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:59.262237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bbf11-1bd2-410b-a4d4-30c29444f560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:59.262992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bbf11-1bd2-410b-a4d4-30c29444f560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:59.264879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bbf11-1bd2-410b-a4d4-30c29444f560 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:59.273234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 949/1911] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:04:59.279869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 963/1912] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:04:59.284247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f847b-131a-4805-b43f-53026faa621d service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:59.294827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f847b-131a-4805-b43f-53026faa621d service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:59.299609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc0157-39cd-4ebf-bcde-1aed360c65f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:59.300656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0157-39cd-4ebf-bcde-1aed360c65f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:59.300942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0157-39cd-4ebf-bcde-1aed360c65f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:59.301215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc0157-39cd-4ebf-bcde-1aed360c65f0 admin admin] PATH_INFO: `/v3/projects/3eb21d6f7d814563bd3d21a2dbb1a404/users/0dadfc4ca22e42e183020ffc572eb90a/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:59.317991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc0157-39cd-4ebf-bcde-1aed360c65f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb21d6f7d814563bd3d21a2dbb1a404, user_id=0dadfc4ca22e42e183020ffc572eb90a, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:59.321343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc0157-39cd-4ebf-bcde-1aed360c65f0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:59.336549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 950/1913] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:59 2022] PUT /identity/v3/projects/3eb21d6f7d814563bd3d21a2dbb1a404/users/0dadfc4ca22e42e183020ffc572eb90a/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:59.341471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bbf11-1bd2-410b-a4d4-30c29444f560 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:59.352523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bbf11-1bd2-410b-a4d4-30c29444f560 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:59.362648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:59.416697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c8dbd-7870-4e2f-acf4-e3eff7ee5bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:59.417359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c8dbd-7870-4e2f-acf4-e3eff7ee5bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:59.417643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c8dbd-7870-4e2f-acf4-e3eff7ee5bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:59.417921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c8dbd-7870-4e2f-acf4-e3eff7ee5bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:59.418688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c8dbd-7870-4e2f-acf4-e3eff7ee5bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:59.420313 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c8dbd-7870-4e2f-acf4-e3eff7ee5bc8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:59.438416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc0157-39cd-4ebf-bcde-1aed360c65f0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:59.438772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 964/1914] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:04:59.449857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc0157-39cd-4ebf-bcde-1aed360c65f0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:59.455569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036b35e-479a-42e8-8fef-9aee89d02d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:59.456242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036b35e-479a-42e8-8fef-9aee89d02d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:59.456527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036b35e-479a-42e8-8fef-9aee89d02d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:59.456807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036b35e-479a-42e8-8fef-9aee89d02d15 admin admin] PATH_INFO: `/v3/projects/3eb21d6f7d814563bd3d21a2dbb1a404/users/0dadfc4ca22e42e183020ffc572eb90a/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:59.459051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036b35e-479a-42e8-8fef-9aee89d02d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb21d6f7d814563bd3d21a2dbb1a404, user_id=0dadfc4ca22e42e183020ffc572eb90a, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:59.462827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036b35e-479a-42e8-8fef-9aee89d02d15 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:59.477582 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 951/1915] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:04:59 2022] PUT /identity/v3/projects/3eb21d6f7d814563bd3d21a2dbb1a404/users/0dadfc4ca22e42e183020ffc572eb90a/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:04:59.488631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aefd0-76ec-4a2a-a36c-3156c4a26f0b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:59.490149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aefd0-76ec-4a2a-a36c-3156c4a26f0b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:59.490149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9aefd0-76ec-4a2a-a36c-3156c4a26f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:59.522060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-fb9aefd0-76ec-4a2a-a36c-3156c4a26f0b None None] MFA Rules not processed for user `0dadfc4ca22e42e183020ffc572eb90a`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:04:59.523658 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fb9aefd0-76ec-4a2a-a36c-3156c4a26f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:59.604161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036b35e-479a-42e8-8fef-9aee89d02d15 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:59.606261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8036b35e-479a-42e8-8fef-9aee89d02d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:59.708601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 965/1916] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:04:59.734365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9aefd0-76ec-4a2a-a36c-3156c4a26f0b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:59.740779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fb9aefd0-76ec-4a2a-a36c-3156c4a26f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:59.765252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036b35e-479a-42e8-8fef-9aee89d02d15 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:59.770979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:59.773492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:59.773779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:59.774059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] PATH_INFO: `/v3/projects/9fead724efd5425691b837316a34b2d3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:59.775362 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fead724efd5425691b837316a34b2d3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:59.777923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:59.779774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9aefd0-76ec-4a2a-a36c-3156c4a26f0b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:04:59.792734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:04:59.793861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:04:59.794178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:04:59.794461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] PATH_INFO: `/v3/users/09d63568f3914675841064cf5528138f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:04:59.799815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d63568f3914675841064cf5528138f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:04:59.804821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:04:59.807378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:04:59.867964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fead724efd5425691b837316a34b2d3', 'request_id': 'req-4954c541-b35d-4d30-82b1-d544467ae1d4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:59.887338 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fead724efd5425691b837316a34b2d3', 'request_id': 'req-4954c541-b35d-4d30-82b1-d544467ae1d4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:59.898359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fead724efd5425691b837316a34b2d3', 'request_id': 'req-4954c541-b35d-4d30-82b1-d544467ae1d4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:59.913896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] The token cache is being invalidate because project 9fead724efd5425691b837316a34b2d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:04:59.914277 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:59.915823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:59.920734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 952/1917] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:04:59 2022] DELETE /identity/v3/projects/9fead724efd5425691b837316a34b2d3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:04:59.920734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:04:59.923086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:04:59.934694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d63568f3914675841064cf5528138f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:59.958879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d63568f3914675841064cf5528138f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:59.959631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d63568f3914675841064cf5528138f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:59.960800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d63568f3914675841064cf5528138f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:04:59.987895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8b570f18-17e3-40b9-9ac2-fb58f63d1fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d63568f3914675841064cf5528138f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.028062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 966/1918] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:04:59 2022] DELETE /identity/v3/users/09d63568f3914675841064cf5528138f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:00.029324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7f2c0-3287-414d-b75d-8f35fac49c02 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:00.029697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7f2c0-3287-414d-b75d-8f35fac49c02 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:00.030819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7f2c0-3287-414d-b75d-8f35fac49c02 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:00.033013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 967/1919] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 11:05:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:05:00.033786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7f2c0-3287-414d-b75d-8f35fac49c02 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:00.042044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-49f7f2c0-3287-414d-b75d-8f35fac49c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:00.057364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:00.077755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954c541-b35d-4d30-82b1-d544467ae1d4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:00.136641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7f2c0-3287-414d-b75d-8f35fac49c02 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:00.142206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:00.142989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:00.143278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:00.143562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] PATH_INFO: `/v3/users/64b183ac0f324a2fafa321eee79744e7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:00.144800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b183ac0f324a2fafa321eee79744e7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:00.146999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:00.169511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c26988-6742-4fb0-9adb-2dab902a3d72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:00.170366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c26988-6742-4fb0-9adb-2dab902a3d72 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:00.171403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c26988-6742-4fb0-9adb-2dab902a3d72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:00.171748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c26988-6742-4fb0-9adb-2dab902a3d72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:00.173665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-21c26988-6742-4fb0-9adb-2dab902a3d72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:00.206370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b183ac0f324a2fafa321eee79744e7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.216591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b183ac0f324a2fafa321eee79744e7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.217633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b183ac0f324a2fafa321eee79744e7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.219132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b183ac0f324a2fafa321eee79744e7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.244260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0b30b4ea-8fe1-427f-beec-8678160df02f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b183ac0f324a2fafa321eee79744e7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.266299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 968/1920] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:00 2022] DELETE /identity/v3/users/64b183ac0f324a2fafa321eee79744e7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:00.266299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae4971-f097-4341-897d-3f6933571f1a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:00.266299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae4971-f097-4341-897d-3f6933571f1a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:00.266299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae4971-f097-4341-897d-3f6933571f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:00.276275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c26988-6742-4fb0-9adb-2dab902a3d72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:00.278735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c26988-6742-4fb0-9adb-2dab902a3d72 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:00.288893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ddae4971-f097-4341-897d-3f6933571f1a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:00.290141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ddae4971-f097-4341-897d-3f6933571f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:00.292008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 953/1921] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:04:59 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:00.293625 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35db85-3f97-4974-8600-177f2ea9a0aa None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:00.293918 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35db85-3f97-4974-8600-177f2ea9a0aa None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:00.294243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35db85-3f97-4974-8600-177f2ea9a0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:00.319269 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0d35db85-3f97-4974-8600-177f2ea9a0aa None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:00.321016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0d35db85-3f97-4974-8600-177f2ea9a0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:00.331319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:00.379612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 969/1922] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:00.395076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae4971-f097-4341-897d-3f6933571f1a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:00.396701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ddae4971-f097-4341-897d-3f6933571f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:00.428121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 954/1923] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:00.428705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35db85-3f97-4974-8600-177f2ea9a0aa None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:00.444066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:00.445508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0d35db85-3f97-4974-8600-177f2ea9a0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:00.480637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae4971-f097-4341-897d-3f6933571f1a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:00.491597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46240c47-52ac-4080-9de1-1d98b6e2f0b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:00.504474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46240c47-52ac-4080-9de1-1d98b6e2f0b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:00.506419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46240c47-52ac-4080-9de1-1d98b6e2f0b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:00.506710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46240c47-52ac-4080-9de1-1d98b6e2f0b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:00.527495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46240c47-52ac-4080-9de1-1d98b6e2f0b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:00.529046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35db85-3f97-4974-8600-177f2ea9a0aa None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:00.529875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46240c47-52ac-4080-9de1-1d98b6e2f0b4 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:00.533001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ca6d8-bceb-4490-bedc-e2398989660d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:00.533863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ca6d8-bceb-4490-bedc-e2398989660d service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:00.534124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ca6d8-bceb-4490-bedc-e2398989660d service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:00.534472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ca6d8-bceb-4490-bedc-e2398989660d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:00.536567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-118ca6d8-bceb-4490-bedc-e2398989660d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:00.545925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 970/1924] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:00.547774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46240c47-52ac-4080-9de1-1d98b6e2f0b4 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:00.563918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-46240c47-52ac-4080-9de1-1d98b6e2f0b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:00.599345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46240c47-52ac-4080-9de1-1d98b6e2f0b4 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:00.609140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f6635-15bc-4cf4-a027-6f5df8726939 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:00.611526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f6635-15bc-4cf4-a027-6f5df8726939 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:00.612131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f6635-15bc-4cf4-a027-6f5df8726939 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:00.612618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f6635-15bc-4cf4-a027-6f5df8726939 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:00.626377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f6635-15bc-4cf4-a027-6f5df8726939 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:00.626618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ca6d8-bceb-4490-bedc-e2398989660d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:00.628270 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ca6d8-bceb-4490-bedc-e2398989660d service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:00.628828 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f6635-15bc-4cf4-a027-6f5df8726939 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:00.644746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 971/1925] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:00.646841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 955/1926] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:00.775909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f6635-15bc-4cf4-a027-6f5df8726939 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:00.779831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3a9f6635-15bc-4cf4-a027-6f5df8726939 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:00.801097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f6635-15bc-4cf4-a027-6f5df8726939 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:00.809384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:00.810606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:00.811254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:00.811978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] PATH_INFO: `/v3/projects/15535ad3ba8149bdb828c30d159c1554` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:00.813881 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15535ad3ba8149bdb828c30d159c1554)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:00.816613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:00.877203 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15535ad3ba8149bdb828c30d159c1554', 'request_id': 'req-f1777214-e293-47cb-a3b4-c6b82c9fa336'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.885911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15535ad3ba8149bdb828c30d159c1554', 'request_id': 'req-f1777214-e293-47cb-a3b4-c6b82c9fa336'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.891968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15535ad3ba8149bdb828c30d159c1554', 'request_id': 'req-f1777214-e293-47cb-a3b4-c6b82c9fa336'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.903655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] The token cache is being invalidate because project 15535ad3ba8149bdb828c30d159c1554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:00.903946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.905005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f1777214-e293-47cb-a3b4-c6b82c9fa336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:00.906936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 972/1927] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:00 2022] DELETE /identity/v3/projects/15535ad3ba8149bdb828c30d159c1554 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:00.912710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2c2e4-e07a-459f-882b-1fe624d45a8a None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:00.913001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2c2e4-e07a-459f-882b-1fe624d45a8a None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:00.913283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2c2e4-e07a-459f-882b-1fe624d45a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:00.943146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-acf2c2e4-e07a-459f-882b-1fe624d45a8a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:00.944414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-acf2c2e4-e07a-459f-882b-1fe624d45a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:00.994381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:01.007385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 956/1928] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:01.015666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1d101-0bdf-4f8b-938c-721576cc5ce3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.016215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1d101-0bdf-4f8b-938c-721576cc5ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.016215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1d101-0bdf-4f8b-938c-721576cc5ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.034985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d4e1d101-0bdf-4f8b-938c-721576cc5ce3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:01.036265 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d4e1d101-0bdf-4f8b-938c-721576cc5ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:01.045474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 973/1929] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:01.050231 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2c2e4-e07a-459f-882b-1fe624d45a8a None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.064906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1d101-0bdf-4f8b-938c-721576cc5ce3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.065037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2c2e4-e07a-459f-882b-1fe624d45a8a None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.067123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d4e1d101-0bdf-4f8b-938c-721576cc5ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:01.071213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3881e-9e01-475a-8cc0-de9e96571a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.071876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3881e-9e01-475a-8cc0-de9e96571a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.072191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3881e-9e01-475a-8cc0-de9e96571a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.072471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3881e-9e01-475a-8cc0-de9e96571a64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.073274 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3881e-9e01-475a-8cc0-de9e96571a64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:01.073622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3881e-9e01-475a-8cc0-de9e96571a64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.076007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3881e-9e01-475a-8cc0-de9e96571a64 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.085261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1d101-0bdf-4f8b-938c-721576cc5ce3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.091600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.093009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.093194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.093671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] PATH_INFO: `/v3/projects/bcf80960d4884720afebaf1b613bdb03` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.095420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcf80960d4884720afebaf1b613bdb03)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.097346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.101520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 957/1930] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:05:01.105987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3881e-9e01-475a-8cc0-de9e96571a64 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.116440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3881e-9e01-475a-8cc0-de9e96571a64 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.121485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd1fd4-3ec3-4981-97ab-d3c12308f04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.122266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd1fd4-3ec3-4981-97ab-d3c12308f04d admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.122621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd1fd4-3ec3-4981-97ab-d3c12308f04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.123004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd1fd4-3ec3-4981-97ab-d3c12308f04d admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.123877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd1fd4-3ec3-4981-97ab-d3c12308f04d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.125875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd1fd4-3ec3-4981-97ab-d3c12308f04d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.156403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcf80960d4884720afebaf1b613bdb03', 'request_id': 'req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.157690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 958/1931] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:01 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:05:01.161862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd1fd4-3ec3-4981-97ab-d3c12308f04d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.165721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcf80960d4884720afebaf1b613bdb03', 'request_id': 'req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.175495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcf80960d4884720afebaf1b613bdb03', 'request_id': 'req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.188801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] The token cache is being invalidate because project bcf80960d4884720afebaf1b613bdb03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:01.189090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.189893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.191798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 974/1932] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:01 2022] DELETE /identity/v3/projects/bcf80960d4884720afebaf1b613bdb03 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:01.195748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.197080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:01.217027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:01.239110 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd1fd4-3ec3-4981-97ab-d3c12308f04d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.244227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc540a88-9058-4223-93f3-b6684c9989a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.245017 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540a88-9058-4223-93f3-b6684c9989a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.245310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540a88-9058-4223-93f3-b6684c9989a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.245574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540a88-9058-4223-93f3-b6684c9989a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.246366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc540a88-9058-4223-93f3-b6684c9989a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.248424 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc540a88-9058-4223-93f3-b6684c9989a3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.274578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 959/1933] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:01 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:05:01.278943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc540a88-9058-4223-93f3-b6684c9989a3 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.280817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dc540a88-9058-4223-93f3-b6684c9989a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:01.288109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4521f-a8b2-4f8f-b8d1-44b82d1202a3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.295124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.296034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.296315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.296589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] PATH_INFO: `/v3/users/1f568768ffe84647ac3f39336e186b1e` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.296715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc540a88-9058-4223-93f3-b6684c9989a3 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.298108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f568768ffe84647ac3f39336e186b1e)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.300348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.313039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fff65-3026-40ec-9a20-cb5d47f61185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.314291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fff65-3026-40ec-9a20-cb5d47f61185 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.315111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fff65-3026-40ec-9a20-cb5d47f61185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.315486 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fff65-3026-40ec-9a20-cb5d47f61185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.316299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fff65-3026-40ec-9a20-cb5d47f61185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.318252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fff65-3026-40ec-9a20-cb5d47f61185 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.339169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 960/1934] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:01.341813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fff65-3026-40ec-9a20-cb5d47f61185 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.355328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fff65-3026-40ec-9a20-cb5d47f61185 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.360770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228815c-52a4-48d4-8932-c40e28b942b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.361499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228815c-52a4-48d4-8932-c40e28b942b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.361784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228815c-52a4-48d4-8932-c40e28b942b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.362063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228815c-52a4-48d4-8932-c40e28b942b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.362963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228815c-52a4-48d4-8932-c40e28b942b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.364781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228815c-52a4-48d4-8932-c40e28b942b8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.368539 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f568768ffe84647ac3f39336e186b1e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.377244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f568768ffe84647ac3f39336e186b1e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.377985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f568768ffe84647ac3f39336e186b1e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.378651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f568768ffe84647ac3f39336e186b1e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.380771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 961/1935] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:05:01.384023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228815c-52a4-48d4-8932-c40e28b942b8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.386442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3228815c-52a4-48d4-8932-c40e28b942b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:01.396809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a426a82a-7ff1-4d4f-b929-a3c18b35e820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f568768ffe84647ac3f39336e186b1e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.401921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228815c-52a4-48d4-8932-c40e28b942b8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.406700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebd30c-fde0-4c4a-8ede-694d6eba22ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.407605 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebd30c-fde0-4c4a-8ede-694d6eba22ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.407887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebd30c-fde0-4c4a-8ede-694d6eba22ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.408160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebd30c-fde0-4c4a-8ede-694d6eba22ff admin admin] PATH_INFO: `/v3/projects/f7932d72b01140ad986815f02cc3020c/users/3f1bcb44702f4344be2601bf06ec4360/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.413774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 975/1936] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:01 2022] DELETE /identity/v3/users/1f568768ffe84647ac3f39336e186b1e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:01.426201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebd30c-fde0-4c4a-8ede-694d6eba22ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7932d72b01140ad986815f02cc3020c, user_id=3f1bcb44702f4344be2601bf06ec4360, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.428890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebd30c-fde0-4c4a-8ede-694d6eba22ff admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.448852 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 962/1937] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:01 2022] PUT /identity/v3/projects/f7932d72b01140ad986815f02cc3020c/users/3f1bcb44702f4344be2601bf06ec4360/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:01.455734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2394-b37d-4a9b-a69c-ce43c0294f57 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.456013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2394-b37d-4a9b-a69c-ce43c0294f57 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.456294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e2394-b37d-4a9b-a69c-ce43c0294f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.484904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d58e2394-b37d-4a9b-a69c-ce43c0294f57 None None] MFA Rules not processed for user `3f1bcb44702f4344be2601bf06ec4360`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:01.486602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d58e2394-b37d-4a9b-a69c-ce43c0294f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:01.566687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 976/1938] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:01.581222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebd30c-fde0-4c4a-8ede-694d6eba22ff admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.583314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-05ebd30c-fde0-4c4a-8ede-694d6eba22ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:01.651488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebd30c-fde0-4c4a-8ede-694d6eba22ff admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.655792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f2e89-b215-44e2-a30d-4b963a4e9794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.656461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2e89-b215-44e2-a30d-4b963a4e9794 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.656745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2e89-b215-44e2-a30d-4b963a4e9794 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.657021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2e89-b215-44e2-a30d-4b963a4e9794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.667317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f2e89-b215-44e2-a30d-4b963a4e9794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.669302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f2e89-b215-44e2-a30d-4b963a4e9794 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.680326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 963/1939] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:01.708520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2394-b37d-4a9b-a69c-ce43c0294f57 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.713520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f2e89-b215-44e2-a30d-4b963a4e9794 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.715778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1b1f2e89-b215-44e2-a30d-4b963a4e9794 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:01.734161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:01.741424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:01.781851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e2394-b37d-4a9b-a69c-ce43c0294f57 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.787108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.787998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.788284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.788562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] PATH_INFO: `/v3/users/3f1bcb44702f4344be2601bf06ec4360` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.789763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f1bcb44702f4344be2601bf06ec4360)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.791962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.795243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f2e89-b215-44e2-a30d-4b963a4e9794 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.800594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:01.801466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.801748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.802070 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] PATH_INFO: `/v3/projects/d0e4e86274c84e40ba656ba040c1d913` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.803291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0e4e86274c84e40ba656ba040c1d913)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:01.805195 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:01.841306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f1bcb44702f4344be2601bf06ec4360'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.848899 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f1bcb44702f4344be2601bf06ec4360'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.849553 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f1bcb44702f4344be2601bf06ec4360'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.850141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f1bcb44702f4344be2601bf06ec4360'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.857436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0e4e86274c84e40ba656ba040c1d913', 'request_id': 'req-8a059d17-152a-4510-b9bb-c3bd564e5505'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.862882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6af8bd0b-cfb1-4c3f-82a3-cfaaee7e8afb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f1bcb44702f4344be2601bf06ec4360'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.865601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0e4e86274c84e40ba656ba040c1d913', 'request_id': 'req-8a059d17-152a-4510-b9bb-c3bd564e5505'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.873824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0e4e86274c84e40ba656ba040c1d913', 'request_id': 'req-8a059d17-152a-4510-b9bb-c3bd564e5505'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.886025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 977/1940] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:01 2022] DELETE /identity/v3/users/3f1bcb44702f4344be2601bf06ec4360 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:01.886428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] The token cache is being invalidate because project d0e4e86274c84e40ba656ba040c1d913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:01.886428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.886592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:01.888802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 964/1941] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:01 2022] DELETE /identity/v3/projects/d0e4e86274c84e40ba656ba040c1d913 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:01.888933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409005c-2714-4dd7-81f0-9c45dbcaa28f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:01.888933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409005c-2714-4dd7-81f0-9c45dbcaa28f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:01.889110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409005c-2714-4dd7-81f0-9c45dbcaa28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:01.892518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.893912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:01.912040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a409005c-2714-4dd7-81f0-9c45dbcaa28f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:01.913320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a409005c-2714-4dd7-81f0-9c45dbcaa28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:01.965331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a059d17-152a-4510-b9bb-c3bd564e5505 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:01.973883 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 978/1942] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:01.984028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409005c-2714-4dd7-81f0-9c45dbcaa28f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:01.985425 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a409005c-2714-4dd7-81f0-9c45dbcaa28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:02.010323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:02.027572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:02.028468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.028753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.029169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] PATH_INFO: `/v3/users/891e740fc1404296b2d98bade0bf18e3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.030464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=891e740fc1404296b2d98bade0bf18e3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:02.032167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:02.032626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:02.044724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409005c-2714-4dd7-81f0-9c45dbcaa28f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:02.049920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325250d-0389-47b5-840e-e60d34b56c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:02.050634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325250d-0389-47b5-840e-e60d34b56c78 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.050906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325250d-0389-47b5-840e-e60d34b56c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.051181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325250d-0389-47b5-840e-e60d34b56c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.060467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325250d-0389-47b5-840e-e60d34b56c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:02.061909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325250d-0389-47b5-840e-e60d34b56c78 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:02.071901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 979/1943] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:02.081620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891e740fc1404296b2d98bade0bf18e3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.082397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891e740fc1404296b2d98bade0bf18e3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.090698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891e740fc1404296b2d98bade0bf18e3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.091331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891e740fc1404296b2d98bade0bf18e3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.098404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891e740fc1404296b2d98bade0bf18e3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.121897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 965/1944] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:01 2022] DELETE /identity/v3/users/891e740fc1404296b2d98bade0bf18e3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:02.413001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325250d-0389-47b5-840e-e60d34b56c78 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:02.415949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f325250d-0389-47b5-840e-e60d34b56c78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:02.430229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325250d-0389-47b5-840e-e60d34b56c78 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:02.493100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:02.495187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:02.496372 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:02.497403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.497777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.498169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] PATH_INFO: `/v3/projects/f7932d72b01140ad986815f02cc3020c` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.499546 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7932d72b01140ad986815f02cc3020c)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:02.501849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:02.516494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a4442-315d-4da6-820a-5f536dff7910 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:02.523910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:02.524795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.525076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.525350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] PATH_INFO: `/v3/projects/bd07684992fd44fab7fb9fe26f67f11c` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.528786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd07684992fd44fab7fb9fe26f67f11c)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:02.547978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:02.584855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7932d72b01140ad986815f02cc3020c', 'request_id': 'req-38c100d9-5104-4d9d-bf30-ea430ca2cc85'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.591398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7932d72b01140ad986815f02cc3020c', 'request_id': 'req-38c100d9-5104-4d9d-bf30-ea430ca2cc85'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.598571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7932d72b01140ad986815f02cc3020c', 'request_id': 'req-38c100d9-5104-4d9d-bf30-ea430ca2cc85'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.611459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] The token cache is being invalidate because project f7932d72b01140ad986815f02cc3020c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:02.611829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.612540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-38c100d9-5104-4d9d-bf30-ea430ca2cc85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.614227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 980/1945] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:02 2022] DELETE /identity/v3/projects/f7932d72b01140ad986815f02cc3020c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:02.618644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd07684992fd44fab7fb9fe26f67f11c', 'request_id': 'req-d660a294-ec6d-4b70-8ff8-396c56cb4823'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.619153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fc05c-961b-4ac9-8573-af32dc16e2f0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.619576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fc05c-961b-4ac9-8573-af32dc16e2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.620001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fc05c-961b-4ac9-8573-af32dc16e2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.637086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd07684992fd44fab7fb9fe26f67f11c', 'request_id': 'req-d660a294-ec6d-4b70-8ff8-396c56cb4823'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.643502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd07684992fd44fab7fb9fe26f67f11c', 'request_id': 'req-d660a294-ec6d-4b70-8ff8-396c56cb4823'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.645938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d60fc05c-961b-4ac9-8573-af32dc16e2f0 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:02.647165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d60fc05c-961b-4ac9-8573-af32dc16e2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:02.656646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] The token cache is being invalidate because project bd07684992fd44fab7fb9fe26f67f11c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:02.657166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.657848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d660a294-ec6d-4b70-8ff8-396c56cb4823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:02.659564 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 966/1946] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:02 2022] DELETE /identity/v3/projects/bd07684992fd44fab7fb9fe26f67f11c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:02.669910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b1aff-a5b6-44d9-8d75-ccebb68ffab0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.671443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b1aff-a5b6-44d9-8d75-ccebb68ffab0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.671443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b1aff-a5b6-44d9-8d75-ccebb68ffab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.692329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-689b1aff-a5b6-44d9-8d75-ccebb68ffab0 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:02.693646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-689b1aff-a5b6-44d9-8d75-ccebb68ffab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:02.718333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 981/1947] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:02.724441 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20314b0e-0fe6-4d8f-b2bf-ed589e627d11 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.726432 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20314b0e-0fe6-4d8f-b2bf-ed589e627d11 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.726894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20314b0e-0fe6-4d8f-b2bf-ed589e627d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.745224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:02.745811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-20314b0e-0fe6-4d8f-b2bf-ed589e627d11 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:02.747161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-20314b0e-0fe6-4d8f-b2bf-ed589e627d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:02.756095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 982/1948] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:02.759829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20314b0e-0fe6-4d8f-b2bf-ed589e627d11 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:02.769344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20314b0e-0fe6-4d8f-b2bf-ed589e627d11 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:02.773966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 967/1949] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:02.774143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b2c71-c8fc-4caa-b7d5-8f972f51ab4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:02.774767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b2c71-c8fc-4caa-b7d5-8f972f51ab4d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.775047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b2c71-c8fc-4caa-b7d5-8f972f51ab4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.775322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b2c71-c8fc-4caa-b7d5-8f972f51ab4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.776085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b2c71-c8fc-4caa-b7d5-8f972f51ab4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:02.776394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b2c71-c8fc-4caa-b7d5-8f972f51ab4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:02.778055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b2c71-c8fc-4caa-b7d5-8f972f51ab4d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:02.782961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f36a1-774e-44d2-be59-07f76f1c88de None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.783253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f36a1-774e-44d2-be59-07f76f1c88de None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.783538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f36a1-774e-44d2-be59-07f76f1c88de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.786730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:02.796148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 983/1950] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:05:02.797214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b2c71-c8fc-4caa-b7d5-8f972f51ab4d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:02.806476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6f4f36a1-774e-44d2-be59-07f76f1c88de None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:02.808077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6f4f36a1-774e-44d2-be59-07f76f1c88de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:02.808746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b2c71-c8fc-4caa-b7d5-8f972f51ab4d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:02.813972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db36553-05f9-4b3c-8d31-e8a52ecfd248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:02.814722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db36553-05f9-4b3c-8d31-e8a52ecfd248 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.814942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db36553-05f9-4b3c-8d31-e8a52ecfd248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.815224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db36553-05f9-4b3c-8d31-e8a52ecfd248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.815920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db36553-05f9-4b3c-8d31-e8a52ecfd248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:02.816761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 968/1951] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:02.817929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db36553-05f9-4b3c-8d31-e8a52ecfd248 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:02.821226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f36a1-774e-44d2-be59-07f76f1c88de None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:02.830792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f36a1-774e-44d2-be59-07f76f1c88de None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:02.835546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54600f-0ece-4d55-ac8e-c810e58be0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:02.836223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54600f-0ece-4d55-ac8e-c810e58be0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.836505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54600f-0ece-4d55-ac8e-c810e58be0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.836853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54600f-0ece-4d55-ac8e-c810e58be0f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.837710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54600f-0ece-4d55-ac8e-c810e58be0f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:02.838230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54600f-0ece-4d55-ac8e-c810e58be0f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:02.840749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54600f-0ece-4d55-ac8e-c810e58be0f1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:02.849807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 984/1952] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:02 2022] POST /identity/v3/projects => generated 359 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:02.854183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db36553-05f9-4b3c-8d31-e8a52ecfd248 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:02.856750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 969/1953] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:05:02.860966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54600f-0ece-4d55-ac8e-c810e58be0f1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:02.872666 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54600f-0ece-4d55-ac8e-c810e58be0f1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:02.928610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db36553-05f9-4b3c-8d31-e8a52ecfd248 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:02.934437 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86d955-ca54-4b0a-874a-cecdead26089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:02.935162 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86d955-ca54-4b0a-874a-cecdead26089 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.935554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86d955-ca54-4b0a-874a-cecdead26089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.935838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86d955-ca54-4b0a-874a-cecdead26089 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.937377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86d955-ca54-4b0a-874a-cecdead26089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:02.938016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108cc81-5e80-4a20-94c1-69e2229fce2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:02.938823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108cc81-5e80-4a20-94c1-69e2229fce2d admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:02.939115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108cc81-5e80-4a20-94c1-69e2229fce2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:02.939323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86d955-ca54-4b0a-874a-cecdead26089 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:02.939416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108cc81-5e80-4a20-94c1-69e2229fce2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:02.940079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108cc81-5e80-4a20-94c1-69e2229fce2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:02.943841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108cc81-5e80-4a20-94c1-69e2229fce2d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:02.966193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 985/1954] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:02 2022] POST /identity/v3/users => generated 280 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:05:02.969070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86d955-ca54-4b0a-874a-cecdead26089 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:02.976999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 970/1955] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:02 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:05:02.978937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108cc81-5e80-4a20-94c1-69e2229fce2d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:02.981221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86d955-ca54-4b0a-874a-cecdead26089 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.047585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f75d4-d6e7-4e96-a169-c29c45505ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.048278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108cc81-5e80-4a20-94c1-69e2229fce2d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.048418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f75d4-d6e7-4e96-a169-c29c45505ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.048682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f75d4-d6e7-4e96-a169-c29c45505ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.049035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f75d4-d6e7-4e96-a169-c29c45505ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.049702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f75d4-d6e7-4e96-a169-c29c45505ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.051605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f75d4-d6e7-4e96-a169-c29c45505ab6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.055371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2c138-84ea-490f-9f1c-72eba29d9ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.056161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2c138-84ea-490f-9f1c-72eba29d9ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.056517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2c138-84ea-490f-9f1c-72eba29d9ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.057433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2c138-84ea-490f-9f1c-72eba29d9ad6 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.058290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2c138-84ea-490f-9f1c-72eba29d9ad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.060348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2c138-84ea-490f-9f1c-72eba29d9ad6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.064872 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 986/1956] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:03.065759 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f75d4-d6e7-4e96-a169-c29c45505ab6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.076296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f75d4-d6e7-4e96-a169-c29c45505ab6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.081243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758bc17-0f41-4516-a3fd-f98cc7ceec7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.081920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758bc17-0f41-4516-a3fd-f98cc7ceec7b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.082234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758bc17-0f41-4516-a3fd-f98cc7ceec7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.082653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758bc17-0f41-4516-a3fd-f98cc7ceec7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.086179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 971/1957] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:02 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:03.086444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758bc17-0f41-4516-a3fd-f98cc7ceec7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.086444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758bc17-0f41-4516-a3fd-f98cc7ceec7b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.087887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2c138-84ea-490f-9f1c-72eba29d9ad6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.099186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 987/1958] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:05:03.099363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2c138-84ea-490f-9f1c-72eba29d9ad6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.100276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758bc17-0f41-4516-a3fd-f98cc7ceec7b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.104014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11365d5-5114-4ecd-b32c-090918d60036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.104658 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11365d5-5114-4ecd-b32c-090918d60036 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.105007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11365d5-5114-4ecd-b32c-090918d60036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.105357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11365d5-5114-4ecd-b32c-090918d60036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.106191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11365d5-5114-4ecd-b32c-090918d60036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.107879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11365d5-5114-4ecd-b32c-090918d60036 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.111352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758bc17-0f41-4516-a3fd-f98cc7ceec7b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.116675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13414c94-d43a-427d-8fe3-f26447539b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.117568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13414c94-d43a-427d-8fe3-f26447539b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.117850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13414c94-d43a-427d-8fe3-f26447539b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.118154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13414c94-d43a-427d-8fe3-f26447539b97 admin admin] PATH_INFO: `/v3/projects/68aab984adfd41c6843c286e54e0ba6a/users/46cb414ba3a4466fa25b1fdfa1a87bf2/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.118884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 972/1959] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:03.122818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11365d5-5114-4ecd-b32c-090918d60036 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.133194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11365d5-5114-4ecd-b32c-090918d60036 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.137994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7d435-4a2f-4111-9d8e-220810cdaaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.138314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13414c94-d43a-427d-8fe3-f26447539b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68aab984adfd41c6843c286e54e0ba6a, user_id=46cb414ba3a4466fa25b1fdfa1a87bf2, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.138726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7d435-4a2f-4111-9d8e-220810cdaaf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.139008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7d435-4a2f-4111-9d8e-220810cdaaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.139286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7d435-4a2f-4111-9d8e-220810cdaaf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.140023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7d435-4a2f-4111-9d8e-220810cdaaf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.141269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13414c94-d43a-427d-8fe3-f26447539b97 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.141737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7d435-4a2f-4111-9d8e-220810cdaaf0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.155100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 973/1960] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:03.159917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7d435-4a2f-4111-9d8e-220810cdaaf0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.169509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 988/1961] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:03 2022] PUT /identity/v3/projects/68aab984adfd41c6843c286e54e0ba6a/users/46cb414ba3a4466fa25b1fdfa1a87bf2/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:03.173960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7d435-4a2f-4111-9d8e-220810cdaaf0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.181699 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edcf5a-5696-4b44-8d64-4ee1ec89f5d2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.182289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edcf5a-5696-4b44-8d64-4ee1ec89f5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.182573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edcf5a-5696-4b44-8d64-4ee1ec89f5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.196268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:03.210854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-22edcf5a-5696-4b44-8d64-4ee1ec89f5d2 None None] MFA Rules not processed for user `46cb414ba3a4466fa25b1fdfa1a87bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:03.212243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-22edcf5a-5696-4b44-8d64-4ee1ec89f5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:03.226458 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:03.251247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e961e-60b4-47f1-820b-6e5ef593e49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.252190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e961e-60b4-47f1-820b-6e5ef593e49d admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.252473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e961e-60b4-47f1-820b-6e5ef593e49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.252757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e961e-60b4-47f1-820b-6e5ef593e49d admin admin] PATH_INFO: `/v3/projects/a75b4648270e429f85a897cd0e489b2f/users/61221e4b415d4886a761efd3de6df158/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.273135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e961e-60b4-47f1-820b-6e5ef593e49d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a75b4648270e429f85a897cd0e489b2f, user_id=61221e4b415d4886a761efd3de6df158, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.276080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e961e-60b4-47f1-820b-6e5ef593e49d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.287057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 989/1962] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:03 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:03.293100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 974/1963] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:03 2022] PUT /identity/v3/projects/a75b4648270e429f85a897cd0e489b2f/users/61221e4b415d4886a761efd3de6df158/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:03.305340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186823b-a3dc-4c07-a7a1-8445df799046 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.305340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186823b-a3dc-4c07-a7a1-8445df799046 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.305635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186823b-a3dc-4c07-a7a1-8445df799046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.324608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e961e-60b4-47f1-820b-6e5ef593e49d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.326504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ef2e961e-60b4-47f1-820b-6e5ef593e49d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:03.334879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0186823b-a3dc-4c07-a7a1-8445df799046 None None] MFA Rules not processed for user `61221e4b415d4886a761efd3de6df158`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:03.336047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0186823b-a3dc-4c07-a7a1-8445df799046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:03.382705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e961e-60b4-47f1-820b-6e5ef593e49d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.386914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bd21f-fbb8-4ffd-8a06-d653ff03ee27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.387573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bd21f-fbb8-4ffd-8a06-d653ff03ee27 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.387964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bd21f-fbb8-4ffd-8a06-d653ff03ee27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.388243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bd21f-fbb8-4ffd-8a06-d653ff03ee27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.402220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 990/1964] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:03.413486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186823b-a3dc-4c07-a7a1-8445df799046 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.414919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0186823b-a3dc-4c07-a7a1-8445df799046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:03.426769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186823b-a3dc-4c07-a7a1-8445df799046 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.430947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a18ad-5991-4dd1-be2b-781b189763b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.431596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a18ad-5991-4dd1-be2b-781b189763b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.431879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a18ad-5991-4dd1-be2b-781b189763b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.432175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a18ad-5991-4dd1-be2b-781b189763b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.440719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a18ad-5991-4dd1-be2b-781b189763b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.442211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a18ad-5991-4dd1-be2b-781b189763b1 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.444648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bd21f-fbb8-4ffd-8a06-d653ff03ee27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.446506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bd21f-fbb8-4ffd-8a06-d653ff03ee27 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.451451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 991/1965] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:03.455486 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 975/1966] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:03 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:03.469489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a18ad-5991-4dd1-be2b-781b189763b1 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.472342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bd21f-fbb8-4ffd-8a06-d653ff03ee27 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.531446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:03.534948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a18ad-5991-4dd1-be2b-781b189763b1 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.535766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bd21f-fbb8-4ffd-8a06-d653ff03ee27 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.540044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.541063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.541394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.541677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] PATH_INFO: `/v3/users/46cb414ba3a4466fa25b1fdfa1a87bf2` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.542101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.542942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46cb414ba3a4466fa25b1fdfa1a87bf2)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.543178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.543574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.543937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] PATH_INFO: `/v3/users/61221e4b415d4886a761efd3de6df158` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.544887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.545368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] RBAC: Authorizing `identity:delete_user(user_id=61221e4b415d4886a761efd3de6df158)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.547632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.591192 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:03.601013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46cb414ba3a4466fa25b1fdfa1a87bf2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:03.601445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61221e4b415d4886a761efd3de6df158'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:03.602291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61221e4b415d4886a761efd3de6df158'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:03.610579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46cb414ba3a4466fa25b1fdfa1a87bf2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:03.612653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46cb414ba3a4466fa25b1fdfa1a87bf2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:03.614662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46cb414ba3a4466fa25b1fdfa1a87bf2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:03.616242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61221e4b415d4886a761efd3de6df158'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:03.617347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61221e4b415d4886a761efd3de6df158'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:03.628992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-e6f20393-3ee8-4156-a27a-ac6bf5cd705e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61221e4b415d4886a761efd3de6df158'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:03.635029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9e49a75e-b92c-4ac0-8b44-cbba36d6c7a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46cb414ba3a4466fa25b1fdfa1a87bf2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:03.656931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 976/1967] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:03 2022] DELETE /identity/v3/users/61221e4b415d4886a761efd3de6df158 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:03.657352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 992/1968] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:03 2022] DELETE /identity/v3/users/46cb414ba3a4466fa25b1fdfa1a87bf2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:03.661430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c25914-7289-40f9-b5e9-9f5a826b6b40 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.661720 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c25914-7289-40f9-b5e9-9f5a826b6b40 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.662001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01882bc-76df-4f41-9873-0079098c07d1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.662317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01882bc-76df-4f41-9873-0079098c07d1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.664636 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01882bc-76df-4f41-9873-0079098c07d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.664836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c25914-7289-40f9-b5e9-9f5a826b6b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.686554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-65c25914-7289-40f9-b5e9-9f5a826b6b40 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:03.687281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e01882bc-76df-4f41-9873-0079098c07d1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:03.688322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-65c25914-7289-40f9-b5e9-9f5a826b6b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:03.688432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e01882bc-76df-4f41-9873-0079098c07d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:03.766770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 977/1969] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:03.780087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01882bc-76df-4f41-9873-0079098c07d1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.783931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e01882bc-76df-4f41-9873-0079098c07d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:03.828345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 993/1970] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:03.842297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c25914-7289-40f9-b5e9-9f5a826b6b40 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:03.844114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-65c25914-7289-40f9-b5e9-9f5a826b6b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:03.900155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01882bc-76df-4f41-9873-0079098c07d1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.905648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec6fe9-9170-4b7a-9585-39c219ca5539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.906715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec6fe9-9170-4b7a-9585-39c219ca5539 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.907068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec6fe9-9170-4b7a-9585-39c219ca5539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.907417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec6fe9-9170-4b7a-9585-39c219ca5539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.919192 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c25914-7289-40f9-b5e9-9f5a826b6b40 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:03.922500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec6fe9-9170-4b7a-9585-39c219ca5539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.923069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f54408d-73b0-463d-94e3-d9d02ed08819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:03.923937 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54408d-73b0-463d-94e3-d9d02ed08819 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:03.924169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54408d-73b0-463d-94e3-d9d02ed08819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:03.924272 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec6fe9-9170-4b7a-9585-39c219ca5539 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.924443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f54408d-73b0-463d-94e3-d9d02ed08819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:03.934315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54408d-73b0-463d-94e3-d9d02ed08819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:03.935464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 978/1971] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:03.938794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f54408d-73b0-463d-94e3-d9d02ed08819 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:03.950009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 994/1972] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:04.291794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec6fe9-9170-4b7a-9585-39c219ca5539 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:04.294244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a2ec6fe9-9170-4b7a-9585-39c219ca5539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:04.362445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec6fe9-9170-4b7a-9585-39c219ca5539 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:04.368845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:04.369775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.370267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.370692 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] PATH_INFO: `/v3/projects/a75b4648270e429f85a897cd0e489b2f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.372133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a75b4648270e429f85a897cd0e489b2f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:04.374150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:04.397277 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:04.457474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a75b4648270e429f85a897cd0e489b2f', 'request_id': 'req-4a83f098-d775-499d-b362-12fc019fa0c6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:04.467914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a75b4648270e429f85a897cd0e489b2f', 'request_id': 'req-4a83f098-d775-499d-b362-12fc019fa0c6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:04.474867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a75b4648270e429f85a897cd0e489b2f', 'request_id': 'req-4a83f098-d775-499d-b362-12fc019fa0c6'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:04.486976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] The token cache is being invalidate because project a75b4648270e429f85a897cd0e489b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:04.487314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:04.487907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:04.490142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 979/1973] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:04 2022] DELETE /identity/v3/projects/a75b4648270e429f85a897cd0e489b2f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:04.496009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b33332-cd5e-42f0-9c30-e85b157ac485 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.496516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b33332-cd5e-42f0-9c30-e85b157ac485 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.496516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b33332-cd5e-42f0-9c30-e85b157ac485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.523037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-22b33332-cd5e-42f0-9c30-e85b157ac485 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:04.524406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-22b33332-cd5e-42f0-9c30-e85b157ac485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:04.604261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:04.618957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:04.620646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:04.627350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 995/1974] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:04.635764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa1986-2dd9-4a1f-b8c5-c56766488423 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.636054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa1986-2dd9-4a1f-b8c5-c56766488423 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.636959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa1986-2dd9-4a1f-b8c5-c56766488423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.640590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f098-d775-499d-b362-12fc019fa0c6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:04.654682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:04.655907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.656329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.656746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] PATH_INFO: `/v3/projects/68aab984adfd41c6843c286e54e0ba6a` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.658169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-93fa1986-2dd9-4a1f-b8c5-c56766488423 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:04.658740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] RBAC: Authorizing `identity:delete_project(project_id=68aab984adfd41c6843c286e54e0ba6a)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:04.659379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-93fa1986-2dd9-4a1f-b8c5-c56766488423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:04.661808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:04.667745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 996/1975] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:04.671969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa1986-2dd9-4a1f-b8c5-c56766488423 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:04.680959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa1986-2dd9-4a1f-b8c5-c56766488423 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:04.686568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7077b9-1ee4-4117-b1a5-616817fe2d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:04.687298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7077b9-1ee4-4117-b1a5-616817fe2d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.687582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7077b9-1ee4-4117-b1a5-616817fe2d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.687852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7077b9-1ee4-4117-b1a5-616817fe2d5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.688609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7077b9-1ee4-4117-b1a5-616817fe2d5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:04.689080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7077b9-1ee4-4117-b1a5-616817fe2d5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:04.690816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7077b9-1ee4-4117-b1a5-616817fe2d5e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:04.706747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 997/1976] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:05:04.711353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7077b9-1ee4-4117-b1a5-616817fe2d5e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:04.721488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7077b9-1ee4-4117-b1a5-616817fe2d5e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:04.721726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68aab984adfd41c6843c286e54e0ba6a', 'request_id': 'req-66c1e143-e7ae-4abf-9f55-1468bad2136f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:04.729829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68aab984adfd41c6843c286e54e0ba6a', 'request_id': 'req-66c1e143-e7ae-4abf-9f55-1468bad2136f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:04.735395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68aab984adfd41c6843c286e54e0ba6a', 'request_id': 'req-66c1e143-e7ae-4abf-9f55-1468bad2136f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:04.747355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] The token cache is being invalidate because project 68aab984adfd41c6843c286e54e0ba6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:04.747762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:04.748501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-66c1e143-e7ae-4abf-9f55-1468bad2136f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:04.750243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 980/1977] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:04 2022] DELETE /identity/v3/projects/68aab984adfd41c6843c286e54e0ba6a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:04.754979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db5690-b37c-4089-b63e-df985dcc04ba None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.755302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db5690-b37c-4089-b63e-df985dcc04ba None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.755577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db5690-b37c-4089-b63e-df985dcc04ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.774646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-54db5690-b37c-4089-b63e-df985dcc04ba None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:04.775969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-54db5690-b37c-4089-b63e-df985dcc04ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:04.778191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10269561-20b5-4636-9c1d-76c9fda833af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:04.778864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10269561-20b5-4636-9c1d-76c9fda833af admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.779143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10269561-20b5-4636-9c1d-76c9fda833af admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.779414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10269561-20b5-4636-9c1d-76c9fda833af admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.780104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10269561-20b5-4636-9c1d-76c9fda833af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:04.782008 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10269561-20b5-4636-9c1d-76c9fda833af admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:04.784073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 981/1978] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:04.793687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910d0f4-dd58-4300-a878-093ab63a8422 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.794062 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910d0f4-dd58-4300-a878-093ab63a8422 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.794432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910d0f4-dd58-4300-a878-093ab63a8422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.814681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 998/1979] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:04 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:04.815809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10269561-20b5-4636-9c1d-76c9fda833af admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:04.817480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0910d0f4-dd58-4300-a878-093ab63a8422 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:04.817697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-10269561-20b5-4636-9c1d-76c9fda833af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:04.818851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0910d0f4-dd58-4300-a878-093ab63a8422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:04.883822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 982/1980] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:04.888083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910d0f4-dd58-4300-a878-093ab63a8422 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:04.889689 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10269561-20b5-4636-9c1d-76c9fda833af admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:04.896144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e98df-50a6-4324-ae24-ff4c0c4a5214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:04.897789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910d0f4-dd58-4300-a878-093ab63a8422 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:04.897947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e98df-50a6-4324-ae24-ff4c0c4a5214 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.898191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e98df-50a6-4324-ae24-ff4c0c4a5214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.898570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e98df-50a6-4324-ae24-ff4c0c4a5214 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.899500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e98df-50a6-4324-ae24-ff4c0c4a5214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:04.906781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e98df-50a6-4324-ae24-ff4c0c4a5214 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:04.915581 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8bff7-41d4-4736-a29b-f7e9d9828fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:04.922918 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8bff7-41d4-4736-a29b-f7e9d9828fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.923385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8bff7-41d4-4736-a29b-f7e9d9828fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.923834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8bff7-41d4-4736-a29b-f7e9d9828fa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.924877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8bff7-41d4-4736-a29b-f7e9d9828fa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:04.925417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8bff7-41d4-4736-a29b-f7e9d9828fa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:04.927333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8bff7-41d4-4736-a29b-f7e9d9828fa2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:04.931398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 999/1981] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:04 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:04.933534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e98df-50a6-4324-ae24-ff4c0c4a5214 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:04.946729 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e98df-50a6-4324-ae24-ff4c0c4a5214 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:04.947135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 983/1982] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:05:04.953122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8bff7-41d4-4736-a29b-f7e9d9828fa2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:04.955470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a03c2-7507-44a8-ad22-c57047871b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:04.956392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a03c2-7507-44a8-ad22-c57047871b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.956644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a03c2-7507-44a8-ad22-c57047871b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.957540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a03c2-7507-44a8-ad22-c57047871b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.957665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a03c2-7507-44a8-ad22-c57047871b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:04.959930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a03c2-7507-44a8-ad22-c57047871b6b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:04.968118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8bff7-41d4-4736-a29b-f7e9d9828fa2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:04.972687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1000/1983] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:04.973059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911437f-2380-42e6-9f81-b494b53bcf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:04.973766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911437f-2380-42e6-9f81-b494b53bcf01 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:04.974049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911437f-2380-42e6-9f81-b494b53bcf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:04.977222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911437f-2380-42e6-9f81-b494b53bcf01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:04.977976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911437f-2380-42e6-9f81-b494b53bcf01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:04.980706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911437f-2380-42e6-9f81-b494b53bcf01 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:04.981582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a03c2-7507-44a8-ad22-c57047871b6b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:04.984815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:04.998496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a03c2-7507-44a8-ad22-c57047871b6b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:05.004442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef5441-de98-4d52-9f39-1d500d5a9c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:05.005167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef5441-de98-4d52-9f39-1d500d5a9c9c admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:05.005461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef5441-de98-4d52-9f39-1d500d5a9c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:05.005738 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef5441-de98-4d52-9f39-1d500d5a9c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:05.006513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef5441-de98-4d52-9f39-1d500d5a9c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:05.008175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef5441-de98-4d52-9f39-1d500d5a9c9c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:05.023151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1001/1984] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:05.023368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 984/1985] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:04 2022] POST /identity/v3/projects => generated 369 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:05.026216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef5441-de98-4d52-9f39-1d500d5a9c9c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:05.026888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911437f-2380-42e6-9f81-b494b53bcf01 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:05.036694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef5441-de98-4d52-9f39-1d500d5a9c9c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:05.068691 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:05.115356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d740cdb-6954-4961-878f-0c93ed6b3c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:05.116269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d740cdb-6954-4961-878f-0c93ed6b3c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:05.116554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d740cdb-6954-4961-878f-0c93ed6b3c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:05.116853 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d740cdb-6954-4961-878f-0c93ed6b3c15 admin admin] PATH_INFO: `/v3/projects/4e8c3aa89da540d8befc024f30db6c0f/users/9e0df5c8c3a041fd96d6d41cac57c350/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:05.117403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911437f-2380-42e6-9f81-b494b53bcf01 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:05.122583 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec44f-9254-49d7-8574-7cb2f1e09293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:05.122913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec44f-9254-49d7-8574-7cb2f1e09293 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:05.123193 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec44f-9254-49d7-8574-7cb2f1e09293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:05.123467 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec44f-9254-49d7-8574-7cb2f1e09293 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:05.124270 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec44f-9254-49d7-8574-7cb2f1e09293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:05.126155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec44f-9254-49d7-8574-7cb2f1e09293 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:05.132444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d740cdb-6954-4961-878f-0c93ed6b3c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8c3aa89da540d8befc024f30db6c0f, user_id=9e0df5c8c3a041fd96d6d41cac57c350, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:05.135212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d740cdb-6954-4961-878f-0c93ed6b3c15 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:05.147863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 985/1986] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:05 2022] POST /identity/v3/users => generated 285 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:05.149087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1002/1987] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:05 2022] PUT /identity/v3/projects/4e8c3aa89da540d8befc024f30db6c0f/users/9e0df5c8c3a041fd96d6d41cac57c350/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:05.152301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec44f-9254-49d7-8574-7cb2f1e09293 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:05.160974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a987-72af-42cc-a273-b2484088f44d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:05.161261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a987-72af-42cc-a273-b2484088f44d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:05.161535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3a987-72af-42cc-a273-b2484088f44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:05.163095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec44f-9254-49d7-8574-7cb2f1e09293 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:05.188457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9bf3a987-72af-42cc-a273-b2484088f44d None None] MFA Rules not processed for user `9e0df5c8c3a041fd96d6d41cac57c350`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:05.189766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9bf3a987-72af-42cc-a273-b2484088f44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:05.223611 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f8d7-c6ba-4e71-820d-9b137d08fbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:05.224379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f8d7-c6ba-4e71-820d-9b137d08fbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:05.224759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f8d7-c6ba-4e71-820d-9b137d08fbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:05.225109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f8d7-c6ba-4e71-820d-9b137d08fbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:05.225958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09f8d7-c6ba-4e71-820d-9b137d08fbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:05.227703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09f8d7-c6ba-4e71-820d-9b137d08fbe5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:05.239103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 986/1988] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:05.243157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f8d7-c6ba-4e71-820d-9b137d08fbe5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:05.252991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f8d7-c6ba-4e71-820d-9b137d08fbe5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:05.257689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb354f57-37ef-4a3f-ae31-a8ca6383f28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:05.258387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb354f57-37ef-4a3f-ae31-a8ca6383f28b admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:05.258677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb354f57-37ef-4a3f-ae31-a8ca6383f28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:05.258956 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb354f57-37ef-4a3f-ae31-a8ca6383f28b admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:05.259684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb354f57-37ef-4a3f-ae31-a8ca6383f28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:05.261383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb354f57-37ef-4a3f-ae31-a8ca6383f28b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:05.264939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1003/1989] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:05.279469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3a987-72af-42cc-a273-b2484088f44d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:05.280452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 987/1990] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:05:05.280949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9bf3a987-72af-42cc-a273-b2484088f44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:05.284694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb354f57-37ef-4a3f-ae31-a8ca6383f28b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:05.295218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb354f57-37ef-4a3f-ae31-a8ca6383f28b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:05.299839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34366d2-6ac9-4b48-bfe9-fce9257b4e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:05.300740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34366d2-6ac9-4b48-bfe9-fce9257b4e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:05.301027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34366d2-6ac9-4b48-bfe9-fce9257b4e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:05.301305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34366d2-6ac9-4b48-bfe9-fce9257b4e10 admin admin] PATH_INFO: `/v3/projects/e0a09ec970db4f6182e567ac40cc6e57/users/2a4f8542e17945ae993984e2b03881c5/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:05.317543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34366d2-6ac9-4b48-bfe9-fce9257b4e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0a09ec970db4f6182e567ac40cc6e57, user_id=2a4f8542e17945ae993984e2b03881c5, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:05.321088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34366d2-6ac9-4b48-bfe9-fce9257b4e10 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:05.335377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 988/1991] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:05 2022] PUT /identity/v3/projects/e0a09ec970db4f6182e567ac40cc6e57/users/2a4f8542e17945ae993984e2b03881c5/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:05.346156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e0eab-334f-4826-b173-f23596578510 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:05.346488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e0eab-334f-4826-b173-f23596578510 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:05.346735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e0eab-334f-4826-b173-f23596578510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:05.347063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3a987-72af-42cc-a273-b2484088f44d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:05.374560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ad1e0eab-334f-4826-b173-f23596578510 None None] MFA Rules not processed for user `2a4f8542e17945ae993984e2b03881c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:05.375789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ad1e0eab-334f-4826-b173-f23596578510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:05.402491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1f133-1673-4bb0-981c-5319f3dd18b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:05.403169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1f133-1673-4bb0-981c-5319f3dd18b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:05.403434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1f133-1673-4bb0-981c-5319f3dd18b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:05.403715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1f133-1673-4bb0-981c-5319f3dd18b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:05.405260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:05.416038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:05.456994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 989/1992] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:05.468877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e0eab-334f-4826-b173-f23596578510 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:05.469139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1f133-1673-4bb0-981c-5319f3dd18b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:05.470502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ad1e0eab-334f-4826-b173-f23596578510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:05.470782 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1f133-1673-4bb0-981c-5319f3dd18b5 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:05.480649 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1004/1993] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:05.532598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e0eab-334f-4826-b173-f23596578510 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:05.536135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b3b9b-0f28-4928-afdf-545625079118 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bf4d9571d541bcbba87094198dc2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:05.536785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b3b9b-0f28-4928-afdf-545625079118 service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:05.537066 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b3b9b-0f28-4928-afdf-545625079118 service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:05.537343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b3b9b-0f28-4928-afdf-545625079118 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:05.546056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b3b9b-0f28-4928-afdf-545625079118 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:05.547525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b3b9b-0f28-4928-afdf-545625079118 service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:05.556259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 990/1994] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:05.957636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1f133-1673-4bb0-981c-5319f3dd18b5 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:05.959311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-feb1f133-1673-4bb0-981c-5319f3dd18b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:06.033857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1f133-1673-4bb0-981c-5319f3dd18b5 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:06.038598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:06.039579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:06.039922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:06.040262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] PATH_INFO: `/v3/users/6b3be1bc8c8c42faae2b5bac34646610` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:06.041677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b3be1bc8c8c42faae2b5bac34646610)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:06.043752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:06.102894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b3be1bc8c8c42faae2b5bac34646610'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:06.123824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b3be1bc8c8c42faae2b5bac34646610'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:06.124437 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b3be1bc8c8c42faae2b5bac34646610'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:06.125093 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b3be1bc8c8c42faae2b5bac34646610'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:06.140967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b3be1bc8c8c42faae2b5bac34646610'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:06.159059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1005/1995] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:05 2022] DELETE /identity/v3/users/6b3be1bc8c8c42faae2b5bac34646610 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:06.165588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b668eec-1f17-4d1f-9627-825c1d9eef97 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:06.166187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b668eec-1f17-4d1f-9627-825c1d9eef97 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:06.166383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b668eec-1f17-4d1f-9627-825c1d9eef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:06.188000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5b668eec-1f17-4d1f-9627-825c1d9eef97 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:06.189498 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5b668eec-1f17-4d1f-9627-825c1d9eef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:06.288828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 991/1996] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:06.308364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:06.308364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:06.368585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:06.396742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ac51e-b145-45ee-9cbd-d6f364f6daae admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:06.400279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf0b42-5df2-41fe-80a3-2ec9559b793b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:06.401149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf0b42-5df2-41fe-80a3-2ec9559b793b service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:06.401430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf0b42-5df2-41fe-80a3-2ec9559b793b service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:06.401707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf0b42-5df2-41fe-80a3-2ec9559b793b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:06.410747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf0b42-5df2-41fe-80a3-2ec9559b793b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:06.412405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf0b42-5df2-41fe-80a3-2ec9559b793b service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:06.422289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1006/1997] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:06.813509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b668eec-1f17-4d1f-9627-825c1d9eef97 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:06.814998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5b668eec-1f17-4d1f-9627-825c1d9eef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:06.841646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b668eec-1f17-4d1f-9627-825c1d9eef97 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:06.847224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:06.848103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:06.848389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:06.848663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] PATH_INFO: `/v3/projects/569e28e736f440fc9337bf843707f261` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:06.849933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=569e28e736f440fc9337bf843707f261)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:06.851921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:06.906799 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '569e28e736f440fc9337bf843707f261', 'request_id': 'req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:06.922309 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '569e28e736f440fc9337bf843707f261', 'request_id': 'req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:06.927656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:06.929001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '569e28e736f440fc9337bf843707f261', 'request_id': 'req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:06.943420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] The token cache is being invalidate because project 569e28e736f440fc9337bf843707f261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:06.943976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:06.944658 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:06.946619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 992/1998] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:06 2022] DELETE /identity/v3/projects/569e28e736f440fc9337bf843707f261 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:06.951438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf0b42-5df2-41fe-80a3-2ec9559b793b service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:06.962606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf0b42-5df2-41fe-80a3-2ec9559b793b service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:07.048122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:07.049027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:07.049312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:07.049594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] PATH_INFO: `/v3/users/0dadfc4ca22e42e183020ffc572eb90a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:07.050893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dadfc4ca22e42e183020ffc572eb90a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:07.052825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:07.109902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dadfc4ca22e42e183020ffc572eb90a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:07.117803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dadfc4ca22e42e183020ffc572eb90a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:07.118505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dadfc4ca22e42e183020ffc572eb90a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:07.119112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dadfc4ca22e42e183020ffc572eb90a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:07.131592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95ed89a6-dab7-4602-b7a0-81a08fd40d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dadfc4ca22e42e183020ffc572eb90a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:07.144573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1007/1999] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:06 2022] DELETE /identity/v3/users/0dadfc4ca22e42e183020ffc572eb90a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:07.528405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:07.530532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:07.545470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b118de8-ab5d-4e66-98aa-fb3e96be36bc admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:07.613695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:07.614648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:07.614930 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:07.615211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] PATH_INFO: `/v3/projects/3eb21d6f7d814563bd3d21a2dbb1a404` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:07.616398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb21d6f7d814563bd3d21a2dbb1a404)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:07.618364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:07.673742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb21d6f7d814563bd3d21a2dbb1a404', 'request_id': 'req-5dc96a56-aadb-4a2a-bb82-1661df177cfa'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:07.689382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb21d6f7d814563bd3d21a2dbb1a404', 'request_id': 'req-5dc96a56-aadb-4a2a-bb82-1661df177cfa'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:07.696278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb21d6f7d814563bd3d21a2dbb1a404', 'request_id': 'req-5dc96a56-aadb-4a2a-bb82-1661df177cfa'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:07.713733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] The token cache is being invalidate because project 3eb21d6f7d814563bd3d21a2dbb1a404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:07.714043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:07.714750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5dc96a56-aadb-4a2a-bb82-1661df177cfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:07.716378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 993/2000] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:07 2022] DELETE /identity/v3/projects/3eb21d6f7d814563bd3d21a2dbb1a404 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:07.721350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90daea64-9b0c-4903-8d00-0b0ffb2c4f91 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:07.721702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90daea64-9b0c-4903-8d00-0b0ffb2c4f91 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:07.722059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90daea64-9b0c-4903-8d00-0b0ffb2c4f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:07.741342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-90daea64-9b0c-4903-8d00-0b0ffb2c4f91 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:07.742692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-90daea64-9b0c-4903-8d00-0b0ffb2c4f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:07.836958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:07.860069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1008/2001] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:07.869567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c840a-1066-4dd4-82e8-947a12dd1f85 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:07.870071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c840a-1066-4dd4-82e8-947a12dd1f85 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:07.870406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c840a-1066-4dd4-82e8-947a12dd1f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:07.919869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-476c840a-1066-4dd4-82e8-947a12dd1f85 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:07.921218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-476c840a-1066-4dd4-82e8-947a12dd1f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:07.931296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 994/2002] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:07.936924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90daea64-9b0c-4903-8d00-0b0ffb2c4f91 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:07.957598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90daea64-9b0c-4903-8d00-0b0ffb2c4f91 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:07.973018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0373a09-37da-4e87-ab21-c39936cef5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:07.973620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0373a09-37da-4e87-ab21-c39936cef5ce admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:07.973620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0373a09-37da-4e87-ab21-c39936cef5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:07.973876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0373a09-37da-4e87-ab21-c39936cef5ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:07.974755 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0373a09-37da-4e87-ab21-c39936cef5ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:07.975223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0373a09-37da-4e87-ab21-c39936cef5ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:07.977237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0373a09-37da-4e87-ab21-c39936cef5ce admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.002336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1009/2003] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:05:08.003850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c840a-1066-4dd4-82e8-947a12dd1f85 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.013879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c840a-1066-4dd4-82e8-947a12dd1f85 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.022882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f4ce0-9b18-4ebe-be46-e7f9063370c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.023569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f4ce0-9b18-4ebe-be46-e7f9063370c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.023849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f4ce0-9b18-4ebe-be46-e7f9063370c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.024127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f4ce0-9b18-4ebe-be46-e7f9063370c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.024915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f4ce0-9b18-4ebe-be46-e7f9063370c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.027059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f4ce0-9b18-4ebe-be46-e7f9063370c9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.062143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 995/2004] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:08 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:08.066781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0373a09-37da-4e87-ab21-c39936cef5ce admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.097999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0373a09-37da-4e87-ab21-c39936cef5ce admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.111559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f4ce0-9b18-4ebe-be46-e7f9063370c9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.113321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ca5f4ce0-9b18-4ebe-be46-e7f9063370c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:08.208210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:08.213087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec598d09-af99-4565-8167-b7f51cf68058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.213939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec598d09-af99-4565-8167-b7f51cf68058 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.214351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec598d09-af99-4565-8167-b7f51cf68058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.214722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec598d09-af99-4565-8167-b7f51cf68058 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.215638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec598d09-af99-4565-8167-b7f51cf68058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.217932 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec598d09-af99-4565-8167-b7f51cf68058 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.234405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f4ce0-9b18-4ebe-be46-e7f9063370c9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.256929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1010/2005] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:08 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:05:08.262126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec598d09-af99-4565-8167-b7f51cf68058 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.264170 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.265087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.265368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.269334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] PATH_INFO: `/v3/users/9e0df5c8c3a041fd96d6d41cac57c350` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.273386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0df5c8c3a041fd96d6d41cac57c350)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.276795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.286594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec598d09-af99-4565-8167-b7f51cf68058 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.292990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbbcba-3f98-4bfd-b373-a06df827fd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.294380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbbcba-3f98-4bfd-b373-a06df827fd14 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.294693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbbcba-3f98-4bfd-b373-a06df827fd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.294980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbbcba-3f98-4bfd-b373-a06df827fd14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.296229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbbcba-3f98-4bfd-b373-a06df827fd14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.297877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbbcba-3f98-4bfd-b373-a06df827fd14 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.312414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1011/2006] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:08.316939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbbcba-3f98-4bfd-b373-a06df827fd14 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.334340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbbcba-3f98-4bfd-b373-a06df827fd14 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.340440 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9b68c-ace8-4a91-890f-48cf1ca843e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.341158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9b68c-ace8-4a91-890f-48cf1ca843e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.341487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9b68c-ace8-4a91-890f-48cf1ca843e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.341766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9b68c-ace8-4a91-890f-48cf1ca843e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.343135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9b68c-ace8-4a91-890f-48cf1ca843e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.344771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9b68c-ace8-4a91-890f-48cf1ca843e0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.362836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1012/2007] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:08.364127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9b68c-ace8-4a91-890f-48cf1ca843e0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.372289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0df5c8c3a041fd96d6d41cac57c350'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:08.378923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0df5c8c3a041fd96d6d41cac57c350'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:08.380168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9b68c-ace8-4a91-890f-48cf1ca843e0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.386110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f6d86-63bd-4a9e-b402-86b453d2dc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.386898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f6d86-63bd-4a9e-b402-86b453d2dc7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.387180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f6d86-63bd-4a9e-b402-86b453d2dc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.387693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f6d86-63bd-4a9e-b402-86b453d2dc7d admin admin] PATH_INFO: `/v3/projects/38c2755b606940c0a7bfdfc0e456b0bc/users/ad5f441ce3a24752bc9f71a97e73dfed/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.407337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0df5c8c3a041fd96d6d41cac57c350'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:08.408502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0df5c8c3a041fd96d6d41cac57c350'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:08.411988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f6d86-63bd-4a9e-b402-86b453d2dc7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c2755b606940c0a7bfdfc0e456b0bc, user_id=ad5f441ce3a24752bc9f71a97e73dfed, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.414804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f6d86-63bd-4a9e-b402-86b453d2dc7d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.421256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d99b1f19-2ea9-4afb-930e-d87c18775724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0df5c8c3a041fd96d6d41cac57c350'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:08.433323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1013/2008] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:08 2022] PUT /identity/v3/projects/38c2755b606940c0a7bfdfc0e456b0bc/users/ad5f441ce3a24752bc9f71a97e73dfed/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:08.445886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 996/2009] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:08 2022] DELETE /identity/v3/users/9e0df5c8c3a041fd96d6d41cac57c350 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:08.446381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99684c7-530d-4e92-913c-884cf5164d24 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.446793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99684c7-530d-4e92-913c-884cf5164d24 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.447685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99684c7-530d-4e92-913c-884cf5164d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.450908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99620eb4-6c70-4548-9a54-ef85c1cef74f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.451455 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99620eb4-6c70-4548-9a54-ef85c1cef74f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.451761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99620eb4-6c70-4548-9a54-ef85c1cef74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.480044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e99684c7-530d-4e92-913c-884cf5164d24 None None] MFA Rules not processed for user `ad5f441ce3a24752bc9f71a97e73dfed`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:08.480903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-99620eb4-6c70-4548-9a54-ef85c1cef74f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:08.481505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e99684c7-530d-4e92-913c-884cf5164d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:08.482219 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-99620eb4-6c70-4548-9a54-ef85c1cef74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:08.564294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 997/2010] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:08.571508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1014/2011] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:08.576226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99620eb4-6c70-4548-9a54-ef85c1cef74f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.577971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-99620eb4-6c70-4548-9a54-ef85c1cef74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:08.578655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99684c7-530d-4e92-913c-884cf5164d24 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.583231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e99684c7-530d-4e92-913c-884cf5164d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:08.591889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99620eb4-6c70-4548-9a54-ef85c1cef74f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.599283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d446961-5ec4-487f-860a-27d23203abd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.600112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d446961-5ec4-487f-860a-27d23203abd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.600395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d446961-5ec4-487f-860a-27d23203abd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.602367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d446961-5ec4-487f-860a-27d23203abd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.603132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d446961-5ec4-487f-860a-27d23203abd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.605437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d446961-5ec4-487f-860a-27d23203abd0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.610614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:08.662123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 998/2012] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:08 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:08.664471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d446961-5ec4-487f-860a-27d23203abd0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.684056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99684c7-530d-4e92-913c-884cf5164d24 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.772301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb3cc-d1d8-486d-818e-9bf0f1fbc44b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.772963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb3cc-d1d8-486d-818e-9bf0f1fbc44b service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.773244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb3cc-d1d8-486d-818e-9bf0f1fbc44b service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.773521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb3cc-d1d8-486d-818e-9bf0f1fbc44b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.778454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:08.792635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d446961-5ec4-487f-860a-27d23203abd0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.801181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c35ab-ca58-42a8-aa84-eefd5915858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.802550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c35ab-ca58-42a8-aa84-eefd5915858e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.803093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c35ab-ca58-42a8-aa84-eefd5915858e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.803558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c35ab-ca58-42a8-aa84-eefd5915858e admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.804389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c35ab-ca58-42a8-aa84-eefd5915858e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.808638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c35ab-ca58-42a8-aa84-eefd5915858e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.838183 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 999/2013] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:08 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:08.840656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c35ab-ca58-42a8-aa84-eefd5915858e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.859948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c35ab-ca58-42a8-aa84-eefd5915858e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.865528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ede0ae-5953-4ef3-84bc-899d4a7bfd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.866303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ede0ae-5953-4ef3-84bc-899d4a7bfd8e admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.866587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ede0ae-5953-4ef3-84bc-899d4a7bfd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.867011 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ede0ae-5953-4ef3-84bc-899d4a7bfd8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.867755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ede0ae-5953-4ef3-84bc-899d4a7bfd8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.869494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ede0ae-5953-4ef3-84bc-899d4a7bfd8e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.874804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbb3cc-d1d8-486d-818e-9bf0f1fbc44b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.877067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbb3cc-d1d8-486d-818e-9bf0f1fbc44b service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.887139 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1000/2014] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:08.889358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ede0ae-5953-4ef3-84bc-899d4a7bfd8e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.899994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1015/2015] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:08.927429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ede0ae-5953-4ef3-84bc-899d4a7bfd8e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:08.937191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a557b-28ff-4577-977d-3465ff008079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:08.938432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a557b-28ff-4577-977d-3465ff008079 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:08.939103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a557b-28ff-4577-977d-3465ff008079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:08.939402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a557b-28ff-4577-977d-3465ff008079 admin admin] PATH_INFO: `/v3/projects/45d1b61e7a10481f9f984ce268e364a0/users/79f730b69c1b4f8896448b079dad2988/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:08.958693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a557b-28ff-4577-977d-3465ff008079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d1b61e7a10481f9f984ce268e364a0, user_id=79f730b69c1b4f8896448b079dad2988, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:08.961672 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a557b-28ff-4577-977d-3465ff008079 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:08.979260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1001/2016] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:08 2022] PUT /identity/v3/projects/45d1b61e7a10481f9f984ce268e364a0/users/79f730b69c1b4f8896448b079dad2988/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:08.984079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb3cc-d1d8-486d-818e-9bf0f1fbc44b service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:08.995433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb3cc-d1d8-486d-818e-9bf0f1fbc44b service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:09.068511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472d0d6-87bc-4b5f-88ef-111a20349e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:09.068511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472d0d6-87bc-4b5f-88ef-111a20349e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:09.068511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472d0d6-87bc-4b5f-88ef-111a20349e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:09.069041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472d0d6-87bc-4b5f-88ef-111a20349e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:09.071322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472d0d6-87bc-4b5f-88ef-111a20349e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:09.073416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472d0d6-87bc-4b5f-88ef-111a20349e8b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:09.110727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1016/2017] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:09.112078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a557b-28ff-4577-977d-3465ff008079 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:09.138331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a557b-28ff-4577-977d-3465ff008079 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:09.148143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fb07a-73ef-4bba-8266-b6d746beff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:09.149771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fb07a-73ef-4bba-8266-b6d746beff97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:09.154504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fb07a-73ef-4bba-8266-b6d746beff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:09.155251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fb07a-73ef-4bba-8266-b6d746beff97 admin admin] PATH_INFO: `/v3/projects/45d1b61e7a10481f9f984ce268e364a0/users/79f730b69c1b4f8896448b079dad2988/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:09.158748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fb07a-73ef-4bba-8266-b6d746beff97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d1b61e7a10481f9f984ce268e364a0, user_id=79f730b69c1b4f8896448b079dad2988, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:09.161447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fb07a-73ef-4bba-8266-b6d746beff97 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:09.176386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1002/2018] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:09 2022] PUT /identity/v3/projects/45d1b61e7a10481f9f984ce268e364a0/users/79f730b69c1b4f8896448b079dad2988/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:09.186924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d180c27-c680-466c-ac77-f6312f249fea None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:09.187241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d180c27-c680-466c-ac77-f6312f249fea None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:09.187489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d180c27-c680-466c-ac77-f6312f249fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:09.223858 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3d180c27-c680-466c-ac77-f6312f249fea None None] MFA Rules not processed for user `79f730b69c1b4f8896448b079dad2988`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:09.225737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3d180c27-c680-466c-ac77-f6312f249fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:09.242017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:09.268480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fb07a-73ef-4bba-8266-b6d746beff97 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:09.270454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-143fb07a-73ef-4bba-8266-b6d746beff97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:09.373929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fb07a-73ef-4bba-8266-b6d746beff97 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:09.374879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1017/2019] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:09.375745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d180c27-c680-466c-ac77-f6312f249fea None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:09.377078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3d180c27-c680-466c-ac77-f6312f249fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:09.385907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:09.387519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:09.387803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:09.388077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] PATH_INFO: `/v3/projects/4e8c3aa89da540d8befc024f30db6c0f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:09.389335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8c3aa89da540d8befc024f30db6c0f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:09.392957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:09.404065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d180c27-c680-466c-ac77-f6312f249fea None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:09.412420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:09.413652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:09.413995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:09.414414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] PATH_INFO: `/v3/users/0e87b5a6574b470bbff04f3949a9cafc` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:09.418639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e87b5a6574b470bbff04f3949a9cafc)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:09.421551 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:09.474079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8c3aa89da540d8befc024f30db6c0f', 'request_id': 'req-574d90a2-06a0-4554-9cdf-07db05a2465b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:09.483629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8c3aa89da540d8befc024f30db6c0f', 'request_id': 'req-574d90a2-06a0-4554-9cdf-07db05a2465b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:09.491736 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8c3aa89da540d8befc024f30db6c0f', 'request_id': 'req-574d90a2-06a0-4554-9cdf-07db05a2465b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:09.508612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e87b5a6574b470bbff04f3949a9cafc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:09.513359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] The token cache is being invalidate because project 4e8c3aa89da540d8befc024f30db6c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:09.513720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:09.514837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:09.517382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1003/2020] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:09 2022] DELETE /identity/v3/projects/4e8c3aa89da540d8befc024f30db6c0f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:09.519016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:09.525541 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:09.532345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e87b5a6574b470bbff04f3949a9cafc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:09.533433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e87b5a6574b470bbff04f3949a9cafc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:09.534526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e87b5a6574b470bbff04f3949a9cafc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:09.554046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-475330e4-4a87-4716-b247-a4bc4fb9c705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e87b5a6574b470bbff04f3949a9cafc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:09.569144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:09.573552 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1018/2021] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:09 2022] DELETE /identity/v3/users/0e87b5a6574b470bbff04f3949a9cafc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:09.579508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6785f-9991-4634-bff9-9dde7fd56459 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:09.581667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6785f-9991-4634-bff9-9dde7fd56459 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:09.582077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6785f-9991-4634-bff9-9dde7fd56459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:09.609638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-95d6785f-9991-4634-bff9-9dde7fd56459 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:09.612549 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-95d6785f-9991-4634-bff9-9dde7fd56459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:09.613850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d90a2-06a0-4554-9cdf-07db05a2465b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:09.689386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351b4f3-83c9-4c11-b5db-2d6c75097c08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:09.690763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351b4f3-83c9-4c11-b5db-2d6c75097c08 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:09.691348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351b4f3-83c9-4c11-b5db-2d6c75097c08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:09.691631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351b4f3-83c9-4c11-b5db-2d6c75097c08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:09.695708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1351b4f3-83c9-4c11-b5db-2d6c75097c08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:09.719363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1019/2022] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:09.729698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6785f-9991-4634-bff9-9dde7fd56459 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:09.730921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-95d6785f-9991-4634-bff9-9dde7fd56459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:09.744704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:09.769660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351b4f3-83c9-4c11-b5db-2d6c75097c08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:09.771417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351b4f3-83c9-4c11-b5db-2d6c75097c08 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:09.781146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1004/2023] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:09.806013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6785f-9991-4634-bff9-9dde7fd56459 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:09.810369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e569e-3959-4464-a469-f43fdf3f0f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:09.811152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e569e-3959-4464-a469-f43fdf3f0f0d service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:09.811513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e569e-3959-4464-a469-f43fdf3f0f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:09.811905 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e569e-3959-4464-a469-f43fdf3f0f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:09.823519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e569e-3959-4464-a469-f43fdf3f0f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:09.825309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e569e-3959-4464-a469-f43fdf3f0f0d service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:09.837747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1020/2024] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:09.936559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351b4f3-83c9-4c11-b5db-2d6c75097c08 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:09.938128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1351b4f3-83c9-4c11-b5db-2d6c75097c08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:09.952447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351b4f3-83c9-4c11-b5db-2d6c75097c08 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:09.957488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:09.958892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:09.959150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:09.959742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] PATH_INFO: `/v3/users/ad5f441ce3a24752bc9f71a97e73dfed` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:09.960664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5f441ce3a24752bc9f71a97e73dfed)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:09.962701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:10.068416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5f441ce3a24752bc9f71a97e73dfed'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.069642 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5f441ce3a24752bc9f71a97e73dfed'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.082200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5f441ce3a24752bc9f71a97e73dfed'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.084205 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5f441ce3a24752bc9f71a97e73dfed'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.094304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5f441ce3a24752bc9f71a97e73dfed'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.113398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:10.132003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1005/2025] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:09 2022] DELETE /identity/v3/users/ad5f441ce3a24752bc9f71a97e73dfed => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:10.133910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217e5e7-31cd-4f1c-b0af-0779da6884e7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:10.134411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217e5e7-31cd-4f1c-b0af-0779da6884e7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:10.134573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217e5e7-31cd-4f1c-b0af-0779da6884e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:10.159804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a217e5e7-31cd-4f1c-b0af-0779da6884e7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:10.161728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a217e5e7-31cd-4f1c-b0af-0779da6884e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:10.223676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:10.226985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:10.280849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1021/2026] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:10.297873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217e5e7-31cd-4f1c-b0af-0779da6884e7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:10.300312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a217e5e7-31cd-4f1c-b0af-0779da6884e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:10.323387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a35c95-c246-4ecc-9ceb-6252ae0cafa9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:10.328191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:10.329101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:10.329390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:10.329669 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] PATH_INFO: `/v3/projects/d9e42229b1a5416c9412da025d51d293` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:10.334639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e42229b1a5416c9412da025d51d293)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:10.334639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:10.380338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217e5e7-31cd-4f1c-b0af-0779da6884e7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:10.394738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e42229b1a5416c9412da025d51d293', 'request_id': 'req-b87fdba3-62a2-44f8-bf19-608b3ac714fa'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.404572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e42229b1a5416c9412da025d51d293', 'request_id': 'req-b87fdba3-62a2-44f8-bf19-608b3ac714fa'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.413517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e42229b1a5416c9412da025d51d293', 'request_id': 'req-b87fdba3-62a2-44f8-bf19-608b3ac714fa'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.432578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] The token cache is being invalidate because project d9e42229b1a5416c9412da025d51d293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:10.433197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.434125 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.435665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1006/2027] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:10 2022] DELETE /identity/v3/projects/d9e42229b1a5416c9412da025d51d293 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:10.439519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:10.440845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:10.441141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb3655-b7e9-4acb-86f2-863bb9a5b15c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:10.441809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb3655-b7e9-4acb-86f2-863bb9a5b15c service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:10.442110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb3655-b7e9-4acb-86f2-863bb9a5b15c service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:10.442440 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb3655-b7e9-4acb-86f2-863bb9a5b15c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:10.444179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-19cb3655-b7e9-4acb-86f2-863bb9a5b15c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:10.477244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:10.511987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fdba3-62a2-44f8-bf19-608b3ac714fa admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:10.515070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb3655-b7e9-4acb-86f2-863bb9a5b15c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:10.516614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb3655-b7e9-4acb-86f2-863bb9a5b15c service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:10.517085 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93008736-707e-4979-b172-07c548930666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:10.518027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93008736-707e-4979-b172-07c548930666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:10.518332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93008736-707e-4979-b172-07c548930666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:10.518614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93008736-707e-4979-b172-07c548930666 admin admin] PATH_INFO: `/v3/users/62756609670b484383b8427321a65bbc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:10.519854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93008736-707e-4979-b172-07c548930666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62756609670b484383b8427321a65bbc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:10.521806 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93008736-707e-4979-b172-07c548930666 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:10.526610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1022/2028] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:10.562051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:10.592328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-93008736-707e-4979-b172-07c548930666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62756609670b484383b8427321a65bbc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.593595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-93008736-707e-4979-b172-07c548930666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62756609670b484383b8427321a65bbc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.604234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-93008736-707e-4979-b172-07c548930666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62756609670b484383b8427321a65bbc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.605120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-93008736-707e-4979-b172-07c548930666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62756609670b484383b8427321a65bbc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.613327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-93008736-707e-4979-b172-07c548930666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62756609670b484383b8427321a65bbc'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:10.631348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1007/2029] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:10 2022] DELETE /identity/v3/users/62756609670b484383b8427321a65bbc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:11.004103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb3655-b7e9-4acb-86f2-863bb9a5b15c service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:11.006408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-19cb3655-b7e9-4acb-86f2-863bb9a5b15c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:11.022194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93008736-707e-4979-b172-07c548930666 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:11.023727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-93008736-707e-4979-b172-07c548930666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:11.033748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb3655-b7e9-4acb-86f2-863bb9a5b15c service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:11.105416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93008736-707e-4979-b172-07c548930666 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:11.110863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:11.111982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.112267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.112823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] PATH_INFO: `/v3/projects/07177cd28e714cd99910e60c2806f994` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.115399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=07177cd28e714cd99910e60c2806f994)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:11.120936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:11.125022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:11.126139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.126254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.126486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] PATH_INFO: `/v3/projects/38c2755b606940c0a7bfdfc0e456b0bc` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.127700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c2755b606940c0a7bfdfc0e456b0bc)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:11.129958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:11.183693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c2755b606940c0a7bfdfc0e456b0bc', 'request_id': 'req-e52a3a3a-8c03-46f9-8fdf-50d232873872'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.191865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07177cd28e714cd99910e60c2806f994', 'request_id': 'req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.195906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c2755b606940c0a7bfdfc0e456b0bc', 'request_id': 'req-e52a3a3a-8c03-46f9-8fdf-50d232873872'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.203088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07177cd28e714cd99910e60c2806f994', 'request_id': 'req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.205387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c2755b606940c0a7bfdfc0e456b0bc', 'request_id': 'req-e52a3a3a-8c03-46f9-8fdf-50d232873872'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.210370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07177cd28e714cd99910e60c2806f994', 'request_id': 'req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.223323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] The token cache is being invalidate because project 07177cd28e714cd99910e60c2806f994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:11.223618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.224241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] The token cache is being invalidate because project 38c2755b606940c0a7bfdfc0e456b0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:11.224468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-175ec2bf-a7ef-4f14-a14f-5bbe387f1d3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.224719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.225601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.226122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1008/2030] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:11 2022] DELETE /identity/v3/projects/07177cd28e714cd99910e60c2806f994 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:11.227463 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1023/2031] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:11 2022] DELETE /identity/v3/projects/38c2755b606940c0a7bfdfc0e456b0bc => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:11.231759 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:11.232185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579b554-7b15-4db9-8f6e-b0788e1afe9d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.232482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579b554-7b15-4db9-8f6e-b0788e1afe9d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.232761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579b554-7b15-4db9-8f6e-b0788e1afe9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.233059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:11.254316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e579b554-7b15-4db9-8f6e-b0788e1afe9d None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:11.255921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e579b554-7b15-4db9-8f6e-b0788e1afe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:11.304812 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:11.309795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:11.334693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a3a3a-8c03-46f9-8fdf-50d232873872 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:11.341361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:11.342447 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.342801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.343083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] PATH_INFO: `/v3/users/79f730b69c1b4f8896448b079dad2988` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.348657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79f730b69c1b4f8896448b079dad2988)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:11.349740 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:11.360697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1009/2032] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:11.368619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe178f12-b7a9-4d98-ba4b-f0f9fd1fb924 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.368619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe178f12-b7a9-4d98-ba4b-f0f9fd1fb924 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.368619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe178f12-b7a9-4d98-ba4b-f0f9fd1fb924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.394680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-fe178f12-b7a9-4d98-ba4b-f0f9fd1fb924 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:11.396334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fe178f12-b7a9-4d98-ba4b-f0f9fd1fb924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:11.406875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1010/2033] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:11.410957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe178f12-b7a9-4d98-ba4b-f0f9fd1fb924 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:11.421642 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe178f12-b7a9-4d98-ba4b-f0f9fd1fb924 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:11.425927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79f730b69c1b4f8896448b079dad2988'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.427336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5287b48-797b-4ef9-8508-376c67d06b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:11.428025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287b48-797b-4ef9-8508-376c67d06b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.428314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287b48-797b-4ef9-8508-376c67d06b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.428855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287b48-797b-4ef9-8508-376c67d06b7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.429632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5287b48-797b-4ef9-8508-376c67d06b7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:11.430436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5287b48-797b-4ef9-8508-376c67d06b7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:11.433237 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5287b48-797b-4ef9-8508-376c67d06b7f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:11.436816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79f730b69c1b4f8896448b079dad2988'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.437590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79f730b69c1b4f8896448b079dad2988'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.438940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79f730b69c1b4f8896448b079dad2988'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.458691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1011/2034] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:05:11.463709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5287b48-797b-4ef9-8508-376c67d06b7f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:11.465947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79f730b69c1b4f8896448b079dad2988'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:11.466307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e5287b48-797b-4ef9-8508-376c67d06b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:11.489624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1024/2035] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:11 2022] DELETE /identity/v3/users/79f730b69c1b4f8896448b079dad2988 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:11.494749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5287b48-797b-4ef9-8508-376c67d06b7f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:11.589361 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d2cf-b6ce-40cb-bb11-d41162be0b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:11.590477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d2cf-b6ce-40cb-bb11-d41162be0b95 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.590808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d2cf-b6ce-40cb-bb11-d41162be0b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.591096 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d2cf-b6ce-40cb-bb11-d41162be0b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.591861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376d2cf-b6ce-40cb-bb11-d41162be0b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:11.593913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376d2cf-b6ce-40cb-bb11-d41162be0b95 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:11.627057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1012/2036] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:11 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:05:11.631508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:11.641917 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17e814-bd41-42aa-8ddb-2f3134e139c7 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:11.708753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c59bb-4f16-4e98-9098-ff79584deb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:11.709765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c59bb-4f16-4e98-9098-ff79584deb37 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.710437 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c59bb-4f16-4e98-9098-ff79584deb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.710825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c59bb-4f16-4e98-9098-ff79584deb37 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.712070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c59bb-4f16-4e98-9098-ff79584deb37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:11.715044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c59bb-4f16-4e98-9098-ff79584deb37 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:11.746167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1025/2037] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:11 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:11.748381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d2cf-b6ce-40cb-bb11-d41162be0b95 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:11.759088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d2cf-b6ce-40cb-bb11-d41162be0b95 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:11.763963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7824b-d83d-49d5-895f-e2ba3b5975e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:11.764709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7824b-d83d-49d5-895f-e2ba3b5975e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.765059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7824b-d83d-49d5-895f-e2ba3b5975e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.765399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7824b-d83d-49d5-895f-e2ba3b5975e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.766246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7824b-d83d-49d5-895f-e2ba3b5975e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:11.767934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7824b-d83d-49d5-895f-e2ba3b5975e4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:11.778931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1013/2038] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:11.783288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c59bb-4f16-4e98-9098-ff79584deb37 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:11.796199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c59bb-4f16-4e98-9098-ff79584deb37 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:11.801195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4be20-4a86-45b2-af7b-c27c1eabc73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:11.802032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4be20-4a86-45b2-af7b-c27c1eabc73c admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.802348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4be20-4a86-45b2-af7b-c27c1eabc73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.802629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4be20-4a86-45b2-af7b-c27c1eabc73c admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.803368 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4be20-4a86-45b2-af7b-c27c1eabc73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:11.805003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4be20-4a86-45b2-af7b-c27c1eabc73c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:11.819747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1026/2039] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:11.825575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7824b-d83d-49d5-895f-e2ba3b5975e4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:11.836839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7824b-d83d-49d5-895f-e2ba3b5975e4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:11.842113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e47c21-02ea-4da4-9ac2-cc5e32d7a165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:11.843041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e47c21-02ea-4da4-9ac2-cc5e32d7a165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.843384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e47c21-02ea-4da4-9ac2-cc5e32d7a165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.843588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e47c21-02ea-4da4-9ac2-cc5e32d7a165 admin admin] PATH_INFO: `/v3/projects/3d5705504a3248979cd7ec79d1fdb4de/users/64d59869f34b4ab08b874feb90fbb349/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.861527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4be20-4a86-45b2-af7b-c27c1eabc73c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:11.866629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1ed4be20-4a86-45b2-af7b-c27c1eabc73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:11.877802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e47c21-02ea-4da4-9ac2-cc5e32d7a165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d5705504a3248979cd7ec79d1fdb4de, user_id=64d59869f34b4ab08b874feb90fbb349, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:11.880536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e47c21-02ea-4da4-9ac2-cc5e32d7a165 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:11.901901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4be20-4a86-45b2-af7b-c27c1eabc73c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:11.903849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1014/2040] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:11 2022] PUT /identity/v3/projects/3d5705504a3248979cd7ec79d1fdb4de/users/64d59869f34b4ab08b874feb90fbb349/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:11.919001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f0c1e-a07e-4211-825a-e059b6658242 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.919397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f0c1e-a07e-4211-825a-e059b6658242 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.919677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f0c1e-a07e-4211-825a-e059b6658242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.952777 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5c2f0c1e-a07e-4211-825a-e059b6658242 None None] MFA Rules not processed for user `64d59869f34b4ab08b874feb90fbb349`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:11.954285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5c2f0c1e-a07e-4211-825a-e059b6658242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:11.973855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:11.987082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:11.988273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:11.989180 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:11.989532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] PATH_INFO: `/v3/projects/45d1b61e7a10481f9f984ce268e364a0` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:11.990891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d1b61e7a10481f9f984ce268e364a0)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:11.993406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.027688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:12.032825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1015/2041] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:12.038814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f0c1e-a07e-4211-825a-e059b6658242 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.051107 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f0c1e-a07e-4211-825a-e059b6658242 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.055070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d1b61e7a10481f9f984ce268e364a0', 'request_id': 'req-5d7ee033-f380-46e3-9abb-37b7c59623e2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:12.065269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d1b61e7a10481f9f984ce268e364a0', 'request_id': 'req-5d7ee033-f380-46e3-9abb-37b7c59623e2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:12.072134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d1b61e7a10481f9f984ce268e364a0', 'request_id': 'req-5d7ee033-f380-46e3-9abb-37b7c59623e2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:12.099363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] The token cache is being invalidate because project 45d1b61e7a10481f9f984ce268e364a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:12.099808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:12.100556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5d7ee033-f380-46e3-9abb-37b7c59623e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:12.102919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1027/2042] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:11 2022] DELETE /identity/v3/projects/45d1b61e7a10481f9f984ce268e364a0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:12.109341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51148f33-8689-4206-b14c-acdb1475699c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.109632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51148f33-8689-4206-b14c-acdb1475699c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.109902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51148f33-8689-4206-b14c-acdb1475699c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.127714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-51148f33-8689-4206-b14c-acdb1475699c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:12.128644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453447f-a887-49c0-9286-1f288a4bf91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.129139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-51148f33-8689-4206-b14c-acdb1475699c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:12.129659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453447f-a887-49c0-9286-1f288a4bf91e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.130147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453447f-a887-49c0-9286-1f288a4bf91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.130589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453447f-a887-49c0-9286-1f288a4bf91e admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.131299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453447f-a887-49c0-9286-1f288a4bf91e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.133241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453447f-a887-49c0-9286-1f288a4bf91e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.137340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1028/2043] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:12.145067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bf841-b23e-4361-a44d-08322c585c59 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.145353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bf841-b23e-4361-a44d-08322c585c59 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.145629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bf841-b23e-4361-a44d-08322c585c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.159973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1016/2044] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:12 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:12.163716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453447f-a887-49c0-9286-1f288a4bf91e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.163832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-167bf841-b23e-4361-a44d-08322c585c59 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:12.164944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-167bf841-b23e-4361-a44d-08322c585c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:12.165291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7453447f-a887-49c0-9286-1f288a4bf91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:12.254406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1029/2045] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:12.255457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bf841-b23e-4361-a44d-08322c585c59 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.258348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453447f-a887-49c0-9286-1f288a4bf91e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.264252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d95ab-925b-495f-b08d-f10c19035e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.264942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d95ab-925b-495f-b08d-f10c19035e44 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.265224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d95ab-925b-495f-b08d-f10c19035e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.265500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d95ab-925b-495f-b08d-f10c19035e44 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.266246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d95ab-925b-495f-b08d-f10c19035e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.267159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bf841-b23e-4361-a44d-08322c585c59 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.268194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d95ab-925b-495f-b08d-f10c19035e44 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.272217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33ff6d-b6cc-48dd-9739-f3cbfbe40612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.272879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33ff6d-b6cc-48dd-9739-f3cbfbe40612 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.273160 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33ff6d-b6cc-48dd-9739-f3cbfbe40612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.273484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33ff6d-b6cc-48dd-9739-f3cbfbe40612 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.274248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33ff6d-b6cc-48dd-9739-f3cbfbe40612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:12.274548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33ff6d-b6cc-48dd-9739-f3cbfbe40612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.277247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33ff6d-b6cc-48dd-9739-f3cbfbe40612 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.294254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1030/2046] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:05:12.295260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33ff6d-b6cc-48dd-9739-f3cbfbe40612 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.300275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1017/2047] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:12 2022] POST /identity/v3/users => generated 297 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:12.301699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d95ab-925b-495f-b08d-f10c19035e44 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.306025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33ff6d-b6cc-48dd-9739-f3cbfbe40612 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.315678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df8313-6136-428b-8484-595f0f209825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.316739 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df8313-6136-428b-8484-595f0f209825 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.317201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df8313-6136-428b-8484-595f0f209825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.317696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df8313-6136-428b-8484-595f0f209825 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.318635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df8313-6136-428b-8484-595f0f209825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.321745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df8313-6136-428b-8484-595f0f209825 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.321878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d95ab-925b-495f-b08d-f10c19035e44 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.328481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bec6c-808d-47ba-9999-d08019d2f88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.329179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bec6c-808d-47ba-9999-d08019d2f88d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.329859 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bec6c-808d-47ba-9999-d08019d2f88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.330182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bec6c-808d-47ba-9999-d08019d2f88d admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.330938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bec6c-808d-47ba-9999-d08019d2f88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.332706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bec6c-808d-47ba-9999-d08019d2f88d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.345580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1018/2048] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:12.349651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bec6c-808d-47ba-9999-d08019d2f88d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.362585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1031/2049] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:12 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:12.362762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bec6c-808d-47ba-9999-d08019d2f88d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.366486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df8313-6136-428b-8484-595f0f209825 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.436498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:12.437322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1537ebb-39d3-4409-8f6a-7156fa5d0dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.439096 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1537ebb-39d3-4409-8f6a-7156fa5d0dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.439507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1537ebb-39d3-4409-8f6a-7156fa5d0dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.439878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1537ebb-39d3-4409-8f6a-7156fa5d0dd1 admin admin] PATH_INFO: `/v3/projects/eea4cc928fc240a59a8238039d9d344d/users/bf725929043343a5a4d76e0771318297/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.451371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:12.459562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1537ebb-39d3-4409-8f6a-7156fa5d0dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea4cc928fc240a59a8238039d9d344d, user_id=bf725929043343a5a4d76e0771318297, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.462239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df8313-6136-428b-8484-595f0f209825 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.462738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1537ebb-39d3-4409-8f6a-7156fa5d0dd1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.467688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30993b56-c7f6-4f82-9401-1c27fe450803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.468503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30993b56-c7f6-4f82-9401-1c27fe450803 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.468864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30993b56-c7f6-4f82-9401-1c27fe450803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.469161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30993b56-c7f6-4f82-9401-1c27fe450803 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.470246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30993b56-c7f6-4f82-9401-1c27fe450803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.472351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30993b56-c7f6-4f82-9401-1c27fe450803 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.487478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1019/2050] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:12 2022] PUT /identity/v3/projects/eea4cc928fc240a59a8238039d9d344d/users/bf725929043343a5a4d76e0771318297/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:12.491567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1537ebb-39d3-4409-8f6a-7156fa5d0dd1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.501267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1032/2051] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:12 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:12.506331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30993b56-c7f6-4f82-9401-1c27fe450803 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.522527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30993b56-c7f6-4f82-9401-1c27fe450803 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.582821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1537ebb-39d3-4409-8f6a-7156fa5d0dd1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.587498 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934c423-894e-493b-9d6f-0e05d2624d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.588431 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934c423-894e-493b-9d6f-0e05d2624d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.589385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934c423-894e-493b-9d6f-0e05d2624d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.589741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934c423-894e-493b-9d6f-0e05d2624d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.590587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934c423-894e-493b-9d6f-0e05d2624d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.592279 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934c423-894e-493b-9d6f-0e05d2624d2c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.607926 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1020/2052] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:05:12.609577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fb8b7-f6e0-42c0-a15e-275b8a8f86d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.609711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934c423-894e-493b-9d6f-0e05d2624d2c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.610461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fb8b7-f6e0-42c0-a15e-275b8a8f86d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.610805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fb8b7-f6e0-42c0-a15e-275b8a8f86d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.611136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fb8b7-f6e0-42c0-a15e-275b8a8f86d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.612145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fb8b7-f6e0-42c0-a15e-275b8a8f86d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.613831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fb8b7-f6e0-42c0-a15e-275b8a8f86d0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.623233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934c423-894e-493b-9d6f-0e05d2624d2c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.627784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62d053-92a2-43f2-a059-ac23b82dbecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.629829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62d053-92a2-43f2-a059-ac23b82dbecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.629829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62d053-92a2-43f2-a059-ac23b82dbecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.629829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62d053-92a2-43f2-a059-ac23b82dbecb admin admin] PATH_INFO: `/v3/projects/eea4cc928fc240a59a8238039d9d344d/users/bf725929043343a5a4d76e0771318297/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.633475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62d053-92a2-43f2-a059-ac23b82dbecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea4cc928fc240a59a8238039d9d344d, user_id=bf725929043343a5a4d76e0771318297, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.634811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1033/2053] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:12.637902 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62d053-92a2-43f2-a059-ac23b82dbecb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.639168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fb8b7-f6e0-42c0-a15e-275b8a8f86d0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.649419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fb8b7-f6e0-42c0-a15e-275b8a8f86d0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.650789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1021/2054] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:12 2022] PUT /identity/v3/projects/eea4cc928fc240a59a8238039d9d344d/users/bf725929043343a5a4d76e0771318297/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:12.661732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107e881-06fa-4fba-a21b-4198bd4a3301 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.662021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107e881-06fa-4fba-a21b-4198bd4a3301 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.662336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107e881-06fa-4fba-a21b-4198bd4a3301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.693142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c107e881-06fa-4fba-a21b-4198bd4a3301 None None] MFA Rules not processed for user `bf725929043343a5a4d76e0771318297`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:12.694451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c107e881-06fa-4fba-a21b-4198bd4a3301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:12.724060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dc1d7-a4bd-48cd-99aa-d0ed8ab8e23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.724980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dc1d7-a4bd-48cd-99aa-d0ed8ab8e23e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.725347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dc1d7-a4bd-48cd-99aa-d0ed8ab8e23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.725620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dc1d7-a4bd-48cd-99aa-d0ed8ab8e23e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.726377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303dc1d7-a4bd-48cd-99aa-d0ed8ab8e23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.728713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303dc1d7-a4bd-48cd-99aa-d0ed8ab8e23e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.742947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1034/2055] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:12.747479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dc1d7-a4bd-48cd-99aa-d0ed8ab8e23e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.764050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dc1d7-a4bd-48cd-99aa-d0ed8ab8e23e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.769066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85bbf82-8ae5-4332-bd83-ee79a9fd4b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.770067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bbf82-8ae5-4332-bd83-ee79a9fd4b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.770474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bbf82-8ae5-4332-bd83-ee79a9fd4b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.770846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85bbf82-8ae5-4332-bd83-ee79a9fd4b9f admin admin] PATH_INFO: `/v3/projects/dd488770726849cf9316c20a84a427f0/users/23966bb025894e2e9011a965c88c742d/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.786060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1022/2056] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:12.787154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bbf82-8ae5-4332-bd83-ee79a9fd4b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd488770726849cf9316c20a84a427f0, user_id=23966bb025894e2e9011a965c88c742d, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.789856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85bbf82-8ae5-4332-bd83-ee79a9fd4b9f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:12.798299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107e881-06fa-4fba-a21b-4198bd4a3301 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.799688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c107e881-06fa-4fba-a21b-4198bd4a3301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:12.803828 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1035/2057] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:12 2022] PUT /identity/v3/projects/dd488770726849cf9316c20a84a427f0/users/23966bb025894e2e9011a965c88c742d/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:12.816141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225e9ec-6820-463f-91f9-47694b75d92a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.821080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225e9ec-6820-463f-91f9-47694b75d92a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.821080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225e9ec-6820-463f-91f9-47694b75d92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.857055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9225e9ec-6820-463f-91f9-47694b75d92a None None] MFA Rules not processed for user `23966bb025894e2e9011a965c88c742d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:12.858758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9225e9ec-6820-463f-91f9-47694b75d92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:12.862997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107e881-06fa-4fba-a21b-4198bd4a3301 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:12.871106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:12.885824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:12.925511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1982e-fa2a-4aeb-9a94-d131bf5b2df4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:12.926244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1982e-fa2a-4aeb-9a94-d131bf5b2df4 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:12.926561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1982e-fa2a-4aeb-9a94-d131bf5b2df4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:12.926842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1982e-fa2a-4aeb-9a94-d131bf5b2df4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:12.933090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1036/2058] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:12.937464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225e9ec-6820-463f-91f9-47694b75d92a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:12.991045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1982e-fa2a-4aeb-9a94-d131bf5b2df4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:12.992633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1982e-fa2a-4aeb-9a94-d131bf5b2df4 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:13.003372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1023/2059] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:13.010873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225e9ec-6820-463f-91f9-47694b75d92a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:13.015408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4c6bd-8a31-435b-aeff-918481a69f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:13.016067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4c6bd-8a31-435b-aeff-918481a69f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:13.016347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4c6bd-8a31-435b-aeff-918481a69f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:13.016618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4c6bd-8a31-435b-aeff-918481a69f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:13.017423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4c6bd-8a31-435b-aeff-918481a69f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:13.019344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4c6bd-8a31-435b-aeff-918481a69f25 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:13.048444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1037/2060] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:12 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:05:13.049636 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1982e-fa2a-4aeb-9a94-d131bf5b2df4 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:13.064318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1982e-fa2a-4aeb-9a94-d131bf5b2df4 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:13.095368 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4c6bd-8a31-435b-aeff-918481a69f25 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:13.139651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69ed9a-78fd-4c00-9b82-0b93cf50e92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:13.140718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69ed9a-78fd-4c00-9b82-0b93cf50e92c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:13.141159 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69ed9a-78fd-4c00-9b82-0b93cf50e92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:13.141538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69ed9a-78fd-4c00-9b82-0b93cf50e92c admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:13.142334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69ed9a-78fd-4c00-9b82-0b93cf50e92c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:13.144600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69ed9a-78fd-4c00-9b82-0b93cf50e92c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:13.165589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4c6bd-8a31-435b-aeff-918481a69f25 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:13.169431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bc283-4f8f-4b9e-b6b1-aa66cb1d3e41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:13.174205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bc283-4f8f-4b9e-b6b1-aa66cb1d3e41 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:13.174205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bc283-4f8f-4b9e-b6b1-aa66cb1d3e41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:13.174205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bc283-4f8f-4b9e-b6b1-aa66cb1d3e41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:13.176599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f42bc283-4f8f-4b9e-b6b1-aa66cb1d3e41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:13.181788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1024/2061] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:13 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:05:13.187078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69ed9a-78fd-4c00-9b82-0b93cf50e92c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:13.201645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69ed9a-78fd-4c00-9b82-0b93cf50e92c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:13.206966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29953e29-1bc3-4e84-aaf0-ee740ff47bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:13.207906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29953e29-1bc3-4e84-aaf0-ee740ff47bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:13.208132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29953e29-1bc3-4e84-aaf0-ee740ff47bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:13.208416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29953e29-1bc3-4e84-aaf0-ee740ff47bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:13.209452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29953e29-1bc3-4e84-aaf0-ee740ff47bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:13.211248 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29953e29-1bc3-4e84-aaf0-ee740ff47bfa admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:13.222359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1025/2062] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:13.226247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29953e29-1bc3-4e84-aaf0-ee740ff47bfa admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:13.236792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29953e29-1bc3-4e84-aaf0-ee740ff47bfa admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:13.242019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba230332-d562-448c-a955-63cfcc31f7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:13.242944 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba230332-d562-448c-a955-63cfcc31f7cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:13.243288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba230332-d562-448c-a955-63cfcc31f7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:13.243499 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba230332-d562-448c-a955-63cfcc31f7cc admin admin] PATH_INFO: `/v3/projects/952f1bb2fec0414882f4ab5f70dd4f9b/users/ef34a60ecb1d4b05be4520d9e22a44a6/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:13.247341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bc283-4f8f-4b9e-b6b1-aa66cb1d3e41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:13.248790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bc283-4f8f-4b9e-b6b1-aa66cb1d3e41 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:13.259592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1038/2063] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:13.260323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba230332-d562-448c-a955-63cfcc31f7cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=952f1bb2fec0414882f4ab5f70dd4f9b, user_id=ef34a60ecb1d4b05be4520d9e22a44a6, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:13.263538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba230332-d562-448c-a955-63cfcc31f7cc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:13.292673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1026/2064] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:13 2022] PUT /identity/v3/projects/952f1bb2fec0414882f4ab5f70dd4f9b/users/ef34a60ecb1d4b05be4520d9e22a44a6/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:13.294455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bc283-4f8f-4b9e-b6b1-aa66cb1d3e41 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:13.315773 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bc283-4f8f-4b9e-b6b1-aa66cb1d3e41 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:13.320611 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:13.404225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710e1c8-942a-4dd1-80d1-d5445c64fed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:13.405021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e1c8-942a-4dd1-80d1-d5445c64fed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:13.405360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e1c8-942a-4dd1-80d1-d5445c64fed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:13.405728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e1c8-942a-4dd1-80d1-d5445c64fed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:13.406933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710e1c8-942a-4dd1-80d1-d5445c64fed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:13.408414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710e1c8-942a-4dd1-80d1-d5445c64fed5 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:13.430043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1039/2065] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:13.434640 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba230332-d562-448c-a955-63cfcc31f7cc admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:13.452300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba230332-d562-448c-a955-63cfcc31f7cc admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:13.464697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e017f63-cf80-4943-8e3a-916b67c91f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:13.466262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e017f63-cf80-4943-8e3a-916b67c91f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:13.466980 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e017f63-cf80-4943-8e3a-916b67c91f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:13.467381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e017f63-cf80-4943-8e3a-916b67c91f15 admin admin] PATH_INFO: `/v3/projects/952f1bb2fec0414882f4ab5f70dd4f9b/users/ef34a60ecb1d4b05be4520d9e22a44a6/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:13.471982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e017f63-cf80-4943-8e3a-916b67c91f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=952f1bb2fec0414882f4ab5f70dd4f9b, user_id=ef34a60ecb1d4b05be4520d9e22a44a6, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:13.476712 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e017f63-cf80-4943-8e3a-916b67c91f15 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:13.494305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1027/2066] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:13 2022] PUT /identity/v3/projects/952f1bb2fec0414882f4ab5f70dd4f9b/users/ef34a60ecb1d4b05be4520d9e22a44a6/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:13.505047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77198ed2-2eab-4a2c-b75d-065ddc4368da None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:13.505572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77198ed2-2eab-4a2c-b75d-065ddc4368da None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:13.505996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77198ed2-2eab-4a2c-b75d-065ddc4368da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:13.584780 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-77198ed2-2eab-4a2c-b75d-065ddc4368da None None] MFA Rules not processed for user `ef34a60ecb1d4b05be4520d9e22a44a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:13.586027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-77198ed2-2eab-4a2c-b75d-065ddc4368da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:13.698052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e017f63-cf80-4943-8e3a-916b67c91f15 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:13.707739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:13.715650 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1040/2067] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:13.734294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77198ed2-2eab-4a2c-b75d-065ddc4368da None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:13.736020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-77198ed2-2eab-4a2c-b75d-065ddc4368da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:13.797138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e017f63-cf80-4943-8e3a-916b67c91f15 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:13.804222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:13.805621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:13.805990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:13.806866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] PATH_INFO: `/v3/users/64d59869f34b4ab08b874feb90fbb349` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:13.808285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d59869f34b4ab08b874feb90fbb349)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:13.810684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:13.849701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77198ed2-2eab-4a2c-b75d-065ddc4368da None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:13.855813 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102c871-0a34-49bb-9840-10cd6a4cfe32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:13.856418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102c871-0a34-49bb-9840-10cd6a4cfe32 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:13.856867 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102c871-0a34-49bb-9840-10cd6a4cfe32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:13.857132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102c871-0a34-49bb-9840-10cd6a4cfe32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:13.868878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102c871-0a34-49bb-9840-10cd6a4cfe32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:13.870644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102c871-0a34-49bb-9840-10cd6a4cfe32 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:13.886218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1041/2068] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:13.923833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d59869f34b4ab08b874feb90fbb349'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:13.923833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d59869f34b4ab08b874feb90fbb349'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:13.939380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d59869f34b4ab08b874feb90fbb349'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:13.940311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d59869f34b4ab08b874feb90fbb349'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:13.961708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d59869f34b4ab08b874feb90fbb349'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:14.020706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1028/2069] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:13 2022] DELETE /identity/v3/users/64d59869f34b4ab08b874feb90fbb349 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:14.025082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5c83-66c8-4f2d-b70c-e625220d9843 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:14.025811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5c83-66c8-4f2d-b70c-e625220d9843 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:14.026440 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f5c83-66c8-4f2d-b70c-e625220d9843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:14.034065 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:14.038758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:14.051900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-544f5c83-66c8-4f2d-b70c-e625220d9843 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:14.053536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-544f5c83-66c8-4f2d-b70c-e625220d9843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:14.104497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:14.165054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1042/2070] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:14.172604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca2417-6963-4d24-8bd1-987b5315cc71 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:14.174375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5c83-66c8-4f2d-b70c-e625220d9843 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:14.178013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-544f5c83-66c8-4f2d-b70c-e625220d9843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:14.180744 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb2ed5-02fc-47d7-8e43-32b03f5a3270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:14.181920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb2ed5-02fc-47d7-8e43-32b03f5a3270 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:14.182761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb2ed5-02fc-47d7-8e43-32b03f5a3270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:14.183316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb2ed5-02fc-47d7-8e43-32b03f5a3270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:14.189610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b4fb2ed5-02fc-47d7-8e43-32b03f5a3270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:14.273809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:14.307811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f5c83-66c8-4f2d-b70c-e625220d9843 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:14.323114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4647d-1f75-4184-9a47-a5ef3b2ceb9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:14.326157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4647d-1f75-4184-9a47-a5ef3b2ceb9e service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:14.328341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4647d-1f75-4184-9a47-a5ef3b2ceb9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:14.328965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4647d-1f75-4184-9a47-a5ef3b2ceb9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:14.337821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb2ed5-02fc-47d7-8e43-32b03f5a3270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:14.340049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb2ed5-02fc-47d7-8e43-32b03f5a3270 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:14.353297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4647d-1f75-4184-9a47-a5ef3b2ceb9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:14.355235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4647d-1f75-4184-9a47-a5ef3b2ceb9e service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:14.362901 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1029/2071] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:14.386834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1043/2072] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:14.798358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb2ed5-02fc-47d7-8e43-32b03f5a3270 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:14.801342 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b4fb2ed5-02fc-47d7-8e43-32b03f5a3270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:14.822744 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb2ed5-02fc-47d7-8e43-32b03f5a3270 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:14.834632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:14.835476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:14.835763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:14.836111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] PATH_INFO: `/v3/projects/3d5705504a3248979cd7ec79d1fdb4de` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:14.837478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d5705504a3248979cd7ec79d1fdb4de)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:14.839476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:14.889045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d5705504a3248979cd7ec79d1fdb4de', 'request_id': 'req-70bfbda5-89c3-4388-9433-41b79287b6ab'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:14.897569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d5705504a3248979cd7ec79d1fdb4de', 'request_id': 'req-70bfbda5-89c3-4388-9433-41b79287b6ab'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:14.903788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d5705504a3248979cd7ec79d1fdb4de', 'request_id': 'req-70bfbda5-89c3-4388-9433-41b79287b6ab'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:14.917280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] The token cache is being invalidate because project 3d5705504a3248979cd7ec79d1fdb4de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:14.917574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:14.918282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:14.919905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1030/2073] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:14 2022] DELETE /identity/v3/projects/3d5705504a3248979cd7ec79d1fdb4de => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:14.924915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4647d-1f75-4184-9a47-a5ef3b2ceb9e service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:14.940701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4647d-1f75-4184-9a47-a5ef3b2ceb9e service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:15.015382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:15.017693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:15.017981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:15.018296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] PATH_INFO: `/v3/users/bf725929043343a5a4d76e0771318297` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:15.020427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf725929043343a5a4d76e0771318297)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:15.022566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:15.087854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf725929043343a5a4d76e0771318297'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:15.097477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf725929043343a5a4d76e0771318297'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:15.098164 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf725929043343a5a4d76e0771318297'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:15.099022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf725929043343a5a4d76e0771318297'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:15.113958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b523e660-99f9-49a6-9775-f54645768e5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf725929043343a5a4d76e0771318297'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:15.120388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:15.131703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1044/2074] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:14 2022] DELETE /identity/v3/users/bf725929043343a5a4d76e0771318297 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:15.459145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:15.461432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:15.477061 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfbda5-89c3-4388-9433-41b79287b6ab admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:15.570919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:15.586891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:15.587216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:15.587550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] PATH_INFO: `/v3/projects/eea4cc928fc240a59a8238039d9d344d` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:15.593937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea4cc928fc240a59a8238039d9d344d)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:15.593937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:15.716567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea4cc928fc240a59a8238039d9d344d', 'request_id': 'req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:15.737531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea4cc928fc240a59a8238039d9d344d', 'request_id': 'req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:15.740028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:15.743406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea4cc928fc240a59a8238039d9d344d', 'request_id': 'req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:15.767606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] The token cache is being invalidate because project eea4cc928fc240a59a8238039d9d344d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:15.772633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:15.774650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-210a39a0-e3e1-46f1-ba5f-10374fb4a0d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:15.781967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1031/2075] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:15 2022] DELETE /identity/v3/projects/eea4cc928fc240a59a8238039d9d344d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:15.784734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350274f5-0f29-4a55-b350-97ca3690a935 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:15.784734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350274f5-0f29-4a55-b350-97ca3690a935 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:15.784734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350274f5-0f29-4a55-b350-97ca3690a935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:15.854195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-350274f5-0f29-4a55-b350-97ca3690a935 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:15.855525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-350274f5-0f29-4a55-b350-97ca3690a935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:16.014372 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1045/2076] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:16.016780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fccea-1182-409f-a31b-ce56ffde4cfd None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:16.017260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fccea-1182-409f-a31b-ce56ffde4cfd None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:16.017479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fccea-1182-409f-a31b-ce56ffde4cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:16.053306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d91fccea-1182-409f-a31b-ce56ffde4cfd None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:16.054569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d91fccea-1182-409f-a31b-ce56ffde4cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:16.062923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1032/2077] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:16.067636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350274f5-0f29-4a55-b350-97ca3690a935 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:16.078515 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350274f5-0f29-4a55-b350-97ca3690a935 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:16.087595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd0b3d-bb37-4e41-b0db-b58280e73e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:16.088449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd0b3d-bb37-4e41-b0db-b58280e73e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:16.088756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd0b3d-bb37-4e41-b0db-b58280e73e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:16.089057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd0b3d-bb37-4e41-b0db-b58280e73e3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:16.089805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd0b3d-bb37-4e41-b0db-b58280e73e3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:16.090148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd0b3d-bb37-4e41-b0db-b58280e73e3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:16.091821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd0b3d-bb37-4e41-b0db-b58280e73e3b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:16.104368 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1046/2078] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:05:16.109377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fccea-1182-409f-a31b-ce56ffde4cfd None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:16.120174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fccea-1182-409f-a31b-ce56ffde4cfd None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:16.125612 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab50a30-b630-423a-8e04-6e8a7232f2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:16.126715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab50a30-b630-423a-8e04-6e8a7232f2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:16.127000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab50a30-b630-423a-8e04-6e8a7232f2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:16.127271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab50a30-b630-423a-8e04-6e8a7232f2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:16.127962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab50a30-b630-423a-8e04-6e8a7232f2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:16.130162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab50a30-b630-423a-8e04-6e8a7232f2b6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:16.157477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1033/2079] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:16 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:16.161572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd0b3d-bb37-4e41-b0db-b58280e73e3b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:16.176808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd0b3d-bb37-4e41-b0db-b58280e73e3b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:16.244946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3bd99-0c49-489e-a577-8c11a1fc7da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:16.245576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3bd99-0c49-489e-a577-8c11a1fc7da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:16.245859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3bd99-0c49-489e-a577-8c11a1fc7da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:16.247155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3bd99-0c49-489e-a577-8c11a1fc7da9 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:16.248041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3bd99-0c49-489e-a577-8c11a1fc7da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:16.250040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3bd99-0c49-489e-a577-8c11a1fc7da9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:16.280422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1047/2080] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:16 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:05:16.282074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab50a30-b630-423a-8e04-6e8a7232f2b6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:16.308997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab50a30-b630-423a-8e04-6e8a7232f2b6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:16.315026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00971588-a20c-4636-9f3c-6bb826332727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:16.315725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00971588-a20c-4636-9f3c-6bb826332727 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:16.316014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00971588-a20c-4636-9f3c-6bb826332727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:16.316288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00971588-a20c-4636-9f3c-6bb826332727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:16.317072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00971588-a20c-4636-9f3c-6bb826332727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:16.318881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00971588-a20c-4636-9f3c-6bb826332727 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:16.331404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1034/2081] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:05:16.335535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3bd99-0c49-489e-a577-8c11a1fc7da9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:16.345712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3bd99-0c49-489e-a577-8c11a1fc7da9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:16.350437 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ae5ba-cc88-425d-a8aa-8dd159d83218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:16.351111 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ae5ba-cc88-425d-a8aa-8dd159d83218 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:16.351400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ae5ba-cc88-425d-a8aa-8dd159d83218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:16.351724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ae5ba-cc88-425d-a8aa-8dd159d83218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:16.352491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ae5ba-cc88-425d-a8aa-8dd159d83218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:16.354386 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ae5ba-cc88-425d-a8aa-8dd159d83218 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:16.365668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1048/2082] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:16.378358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00971588-a20c-4636-9f3c-6bb826332727 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:16.397867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00971588-a20c-4636-9f3c-6bb826332727 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:16.404150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d53c9a-aa17-4460-80ab-337e23630b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:16.405033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d53c9a-aa17-4460-80ab-337e23630b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:16.405563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d53c9a-aa17-4460-80ab-337e23630b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:16.406006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d53c9a-aa17-4460-80ab-337e23630b1e admin admin] PATH_INFO: `/v3/projects/d27dc6069a79488890a6c6b1cda04d97/users/cc6583c280f54a7d92907820d5700623/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:16.431890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d53c9a-aa17-4460-80ab-337e23630b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27dc6069a79488890a6c6b1cda04d97, user_id=cc6583c280f54a7d92907820d5700623, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:16.435085 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d53c9a-aa17-4460-80ab-337e23630b1e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:16.450389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1035/2083] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:16 2022] PUT /identity/v3/projects/d27dc6069a79488890a6c6b1cda04d97/users/cc6583c280f54a7d92907820d5700623/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:16.460928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502376e4-9eed-42ac-bd8b-63708c35e341 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:16.461225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502376e4-9eed-42ac-bd8b-63708c35e341 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:16.461507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502376e4-9eed-42ac-bd8b-63708c35e341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:16.491397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-502376e4-9eed-42ac-bd8b-63708c35e341 None None] MFA Rules not processed for user `cc6583c280f54a7d92907820d5700623`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:16.492616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-502376e4-9eed-42ac-bd8b-63708c35e341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:16.540049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:16.562143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1049/2084] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:16.573996 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d53c9a-aa17-4460-80ab-337e23630b1e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:16.576107 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-75d53c9a-aa17-4460-80ab-337e23630b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:16.641946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d53c9a-aa17-4460-80ab-337e23630b1e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:16.646147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c61872-f5a3-4f44-b108-1de1f7f04476 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:16.646816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61872-f5a3-4f44-b108-1de1f7f04476 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:16.647097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61872-f5a3-4f44-b108-1de1f7f04476 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:16.647406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61872-f5a3-4f44-b108-1de1f7f04476 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:16.656870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c61872-f5a3-4f44-b108-1de1f7f04476 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:16.659076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c61872-f5a3-4f44-b108-1de1f7f04476 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:16.669321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1036/2085] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:19.632266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502376e4-9eed-42ac-bd8b-63708c35e341 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:19.634680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-502376e4-9eed-42ac-bd8b-63708c35e341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:19.714384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502376e4-9eed-42ac-bd8b-63708c35e341 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:19.719547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:19.720562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:19.720849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:19.721126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] PATH_INFO: `/v3/users/23966bb025894e2e9011a965c88c742d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:19.722364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=23966bb025894e2e9011a965c88c742d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:19.724431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:19.788499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23966bb025894e2e9011a965c88c742d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:19.797936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23966bb025894e2e9011a965c88c742d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:19.799656 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23966bb025894e2e9011a965c88c742d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:19.800286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23966bb025894e2e9011a965c88c742d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:19.815270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23966bb025894e2e9011a965c88c742d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:19.838631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1050/2086] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:19 2022] DELETE /identity/v3/users/23966bb025894e2e9011a965c88c742d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:19.843680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bce10-8330-4c05-af3c-b655b6af9247 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:19.844189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bce10-8330-4c05-af3c-b655b6af9247 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:19.844302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bce10-8330-4c05-af3c-b655b6af9247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:19.872882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-984bce10-8330-4c05-af3c-b655b6af9247 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:19.874250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-984bce10-8330-4c05-af3c-b655b6af9247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:19.949423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1037/2087] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:19.956521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:19.958068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:20.026356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b922b-196a-4d01-8eb3-1598faa267fe admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:20.030668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091c13-5600-4640-865b-5e11261ed81e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:20.031333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091c13-5600-4640-865b-5e11261ed81e service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:20.031612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091c13-5600-4640-865b-5e11261ed81e service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:20.031884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091c13-5600-4640-865b-5e11261ed81e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:20.042692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28091c13-5600-4640-865b-5e11261ed81e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:20.044310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28091c13-5600-4640-865b-5e11261ed81e service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:20.054416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1051/2088] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:20.343141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bce10-8330-4c05-af3c-b655b6af9247 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:20.345391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-984bce10-8330-4c05-af3c-b655b6af9247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:20.367501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bce10-8330-4c05-af3c-b655b6af9247 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:20.373753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:20.375052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:20.375354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:20.375766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] PATH_INFO: `/v3/projects/dd488770726849cf9316c20a84a427f0` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:20.377148 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd488770726849cf9316c20a84a427f0)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:20.379152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:20.404956 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:20.432868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd488770726849cf9316c20a84a427f0', 'request_id': 'req-ed11e5e3-b83b-4a97-a542-4cddc445a31a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:20.441304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd488770726849cf9316c20a84a427f0', 'request_id': 'req-ed11e5e3-b83b-4a97-a542-4cddc445a31a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:20.447765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd488770726849cf9316c20a84a427f0', 'request_id': 'req-ed11e5e3-b83b-4a97-a542-4cddc445a31a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:20.462513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] The token cache is being invalidate because project dd488770726849cf9316c20a84a427f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:20.462798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:20.463525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:20.465189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1038/2089] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:20 2022] DELETE /identity/v3/projects/dd488770726849cf9316c20a84a427f0 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:20.469286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091c13-5600-4640-865b-5e11261ed81e service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:20.479665 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091c13-5600-4640-865b-5e11261ed81e service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:20.488332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:20.539558 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:20.540438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:20.540721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:20.541036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] PATH_INFO: `/v3/users/ef34a60ecb1d4b05be4520d9e22a44a6` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:20.542291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef34a60ecb1d4b05be4520d9e22a44a6)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:20.544233 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:20.595035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef34a60ecb1d4b05be4520d9e22a44a6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:20.604358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef34a60ecb1d4b05be4520d9e22a44a6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:20.604981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef34a60ecb1d4b05be4520d9e22a44a6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:20.605604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef34a60ecb1d4b05be4520d9e22a44a6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:20.617493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-16e14ba5-963b-4696-8641-c1a81085f091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef34a60ecb1d4b05be4520d9e22a44a6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:20.630119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1052/2090] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:20 2022] DELETE /identity/v3/users/ef34a60ecb1d4b05be4520d9e22a44a6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:20.916840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:20.919311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:20.935985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11e5e3-b83b-4a97-a542-4cddc445a31a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.001204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.002081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.002394 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.002663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] PATH_INFO: `/v3/projects/952f1bb2fec0414882f4ab5f70dd4f9b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.003846 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=952f1bb2fec0414882f4ab5f70dd4f9b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.005792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.055150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '952f1bb2fec0414882f4ab5f70dd4f9b', 'request_id': 'req-b75cee5a-4c5c-4a10-b438-d64f684e1b58'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:21.064750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '952f1bb2fec0414882f4ab5f70dd4f9b', 'request_id': 'req-b75cee5a-4c5c-4a10-b438-d64f684e1b58'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:21.070722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '952f1bb2fec0414882f4ab5f70dd4f9b', 'request_id': 'req-b75cee5a-4c5c-4a10-b438-d64f684e1b58'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:21.081868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] The token cache is being invalidate because project 952f1bb2fec0414882f4ab5f70dd4f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:21.082188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:21.082861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b75cee5a-4c5c-4a10-b438-d64f684e1b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:21.084409 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1039/2091] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:20 2022] DELETE /identity/v3/projects/952f1bb2fec0414882f4ab5f70dd4f9b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:21.090983 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90f158-9936-40a8-a6e5-4696905ab944 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.091440 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90f158-9936-40a8-a6e5-4696905ab944 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.091526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90f158-9936-40a8-a6e5-4696905ab944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.109765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4b90f158-9936-40a8-a6e5-4696905ab944 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:21.110997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4b90f158-9936-40a8-a6e5-4696905ab944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:21.173723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1053/2092] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:21.182793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400c38-42c9-449f-9633-53ff1207a3ec None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.183284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400c38-42c9-449f-9633-53ff1207a3ec None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.183600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c400c38-42c9-449f-9633-53ff1207a3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.205488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7c400c38-42c9-449f-9633-53ff1207a3ec None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:21.206788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7c400c38-42c9-449f-9633-53ff1207a3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:21.216838 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1040/2093] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:21.221779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90f158-9936-40a8-a6e5-4696905ab944 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.232012 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90f158-9936-40a8-a6e5-4696905ab944 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.237154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a193b-56c2-4e20-b1ef-58d15d023954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.237814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a193b-56c2-4e20-b1ef-58d15d023954 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.238118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a193b-56c2-4e20-b1ef-58d15d023954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.238402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a193b-56c2-4e20-b1ef-58d15d023954 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.239129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a193b-56c2-4e20-b1ef-58d15d023954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:21.239439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a193b-56c2-4e20-b1ef-58d15d023954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.241041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a193b-56c2-4e20-b1ef-58d15d023954 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.252863 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1054/2094] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:05:21.257315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400c38-42c9-449f-9633-53ff1207a3ec None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.267118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c400c38-42c9-449f-9633-53ff1207a3ec None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.272561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1c9a0-a027-482f-a13b-4574a33caf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.273251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1c9a0-a027-482f-a13b-4574a33caf3a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.273561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1c9a0-a027-482f-a13b-4574a33caf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.273877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1c9a0-a027-482f-a13b-4574a33caf3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.274643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1c9a0-a027-482f-a13b-4574a33caf3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.276579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1c9a0-a027-482f-a13b-4574a33caf3a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.303579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1041/2095] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:21 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:21.308135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a193b-56c2-4e20-b1ef-58d15d023954 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.319538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a193b-56c2-4e20-b1ef-58d15d023954 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.334014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:21.381564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fef42-a7f1-48fa-a50d-da9a1ce87ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.382403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fef42-a7f1-48fa-a50d-da9a1ce87ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.382687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fef42-a7f1-48fa-a50d-da9a1ce87ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.382955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fef42-a7f1-48fa-a50d-da9a1ce87ffd admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.383647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fef42-a7f1-48fa-a50d-da9a1ce87ffd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.385594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fef42-a7f1-48fa-a50d-da9a1ce87ffd admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.408889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1055/2096] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:21 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:21.413215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1c9a0-a027-482f-a13b-4574a33caf3a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.424311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1c9a0-a027-482f-a13b-4574a33caf3a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.429242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0d474-8c4c-40dd-88ed-eb5b65bdcfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.429916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0d474-8c4c-40dd-88ed-eb5b65bdcfac admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.430228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0d474-8c4c-40dd-88ed-eb5b65bdcfac admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.430510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0d474-8c4c-40dd-88ed-eb5b65bdcfac admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.431253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0d474-8c4c-40dd-88ed-eb5b65bdcfac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.432939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d0d474-8c4c-40dd-88ed-eb5b65bdcfac admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.445718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1042/2097] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:21.450750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fef42-a7f1-48fa-a50d-da9a1ce87ffd admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.466449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fef42-a7f1-48fa-a50d-da9a1ce87ffd admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.472078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859354ff-9031-4a9b-abc2-36dd80d79d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.472755 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859354ff-9031-4a9b-abc2-36dd80d79d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.473066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859354ff-9031-4a9b-abc2-36dd80d79d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.473345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859354ff-9031-4a9b-abc2-36dd80d79d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.474111 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859354ff-9031-4a9b-abc2-36dd80d79d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.475774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859354ff-9031-4a9b-abc2-36dd80d79d7b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.486496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1056/2098] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:21.490675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0d474-8c4c-40dd-88ed-eb5b65bdcfac admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.495947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:21.502287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0d474-8c4c-40dd-88ed-eb5b65bdcfac admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.507924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccf51b-99b2-4525-9a06-b1481c73fdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.508808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccf51b-99b2-4525-9a06-b1481c73fdae admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.509092 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccf51b-99b2-4525-9a06-b1481c73fdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.509362 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccf51b-99b2-4525-9a06-b1481c73fdae admin admin] PATH_INFO: `/v3/projects/7eccdb52f54f4a3db742b60e9bef9cff/users/162afb114d494d8ab1280b912bd9cc22/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.524554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccf51b-99b2-4525-9a06-b1481c73fdae admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eccdb52f54f4a3db742b60e9bef9cff, user_id=162afb114d494d8ab1280b912bd9cc22, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.527399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccf51b-99b2-4525-9a06-b1481c73fdae admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.543217 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1043/2099] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:21 2022] PUT /identity/v3/projects/7eccdb52f54f4a3db742b60e9bef9cff/users/162afb114d494d8ab1280b912bd9cc22/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:21.553431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2393c-76e0-4410-8d7d-3669c09df6fa None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.553795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2393c-76e0-4410-8d7d-3669c09df6fa None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.554106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2393c-76e0-4410-8d7d-3669c09df6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.582216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d3b2393c-76e0-4410-8d7d-3669c09df6fa None None] MFA Rules not processed for user `162afb114d494d8ab1280b912bd9cc22`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:21.583469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d3b2393c-76e0-4410-8d7d-3669c09df6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:21.650454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1057/2100] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:21.655119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccf51b-99b2-4525-9a06-b1481c73fdae admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.719199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccf51b-99b2-4525-9a06-b1481c73fdae admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.724274 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50175be2-5137-4d85-a111-6815944aa674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.724938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50175be2-5137-4d85-a111-6815944aa674 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.725215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50175be2-5137-4d85-a111-6815944aa674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.725502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50175be2-5137-4d85-a111-6815944aa674 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.726218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50175be2-5137-4d85-a111-6815944aa674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.728148 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50175be2-5137-4d85-a111-6815944aa674 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.761758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1044/2101] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:21 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:21.766362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2393c-76e0-4410-8d7d-3669c09df6fa None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.781775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2393c-76e0-4410-8d7d-3669c09df6fa None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.840598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab53138-3c4d-4440-942a-76e63782808c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.841270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab53138-3c4d-4440-942a-76e63782808c admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.841551 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab53138-3c4d-4440-942a-76e63782808c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.841829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab53138-3c4d-4440-942a-76e63782808c admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.842568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab53138-3c4d-4440-942a-76e63782808c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.844521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab53138-3c4d-4440-942a-76e63782808c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.866172 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1058/2102] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:21 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:21.869986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50175be2-5137-4d85-a111-6815944aa674 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.880809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50175be2-5137-4d85-a111-6815944aa674 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.886285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3bdc7-9e3f-47d8-b837-e22fc897f5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.887038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3bdc7-9e3f-47d8-b837-e22fc897f5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.887319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3bdc7-9e3f-47d8-b837-e22fc897f5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.887592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3bdc7-9e3f-47d8-b837-e22fc897f5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.888328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3bdc7-9e3f-47d8-b837-e22fc897f5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.889944 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3bdc7-9e3f-47d8-b837-e22fc897f5d1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.900578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1045/2103] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:21.905374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab53138-3c4d-4440-942a-76e63782808c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.916277 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab53138-3c4d-4440-942a-76e63782808c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:21.921655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b45a1a-e5b0-4784-80e7-3b8a96f487af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:21.922592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b45a1a-e5b0-4784-80e7-3b8a96f487af admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:21.922881 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b45a1a-e5b0-4784-80e7-3b8a96f487af admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:21.923156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b45a1a-e5b0-4784-80e7-3b8a96f487af admin admin] PATH_INFO: `/v3/projects/184b39e6824f4942bfa255cd7324e873/users/9915b1158d3f45a6a143efd8eb95bdee/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:21.937598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:21.942429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b45a1a-e5b0-4784-80e7-3b8a96f487af admin admin] RBAC: Authorizing `identity:create_grant(project_id=184b39e6824f4942bfa255cd7324e873, user_id=9915b1158d3f45a6a143efd8eb95bdee, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:21.946452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b45a1a-e5b0-4784-80e7-3b8a96f487af admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:21.962113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1059/2104] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:21 2022] PUT /identity/v3/projects/184b39e6824f4942bfa255cd7324e873/users/9915b1158d3f45a6a143efd8eb95bdee/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:21.967589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3bdc7-9e3f-47d8-b837-e22fc897f5d1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:21.978240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3bdc7-9e3f-47d8-b837-e22fc897f5d1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:22.049218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59efde-4627-49e3-bbc8-c1dec3d69c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:22.049892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59efde-4627-49e3-bbc8-c1dec3d69c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:22.050218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59efde-4627-49e3-bbc8-c1dec3d69c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:22.050495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59efde-4627-49e3-bbc8-c1dec3d69c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:22.051226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59efde-4627-49e3-bbc8-c1dec3d69c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:22.052852 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59efde-4627-49e3-bbc8-c1dec3d69c22 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:22.064277 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1046/2105] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:22.068556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b45a1a-e5b0-4784-80e7-3b8a96f487af admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:22.079985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b45a1a-e5b0-4784-80e7-3b8a96f487af admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:22.085531 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315554ae-c05f-48d6-b7ff-87b38821b880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:22.086469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315554ae-c05f-48d6-b7ff-87b38821b880 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:22.086754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315554ae-c05f-48d6-b7ff-87b38821b880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:22.087026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315554ae-c05f-48d6-b7ff-87b38821b880 admin admin] PATH_INFO: `/v3/projects/184b39e6824f4942bfa255cd7324e873/users/9915b1158d3f45a6a143efd8eb95bdee/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:22.089400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315554ae-c05f-48d6-b7ff-87b38821b880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=184b39e6824f4942bfa255cd7324e873, user_id=9915b1158d3f45a6a143efd8eb95bdee, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:22.092322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315554ae-c05f-48d6-b7ff-87b38821b880 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:22.104502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1060/2106] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:22 2022] PUT /identity/v3/projects/184b39e6824f4942bfa255cd7324e873/users/9915b1158d3f45a6a143efd8eb95bdee/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:22.115184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd278ef7-0f4d-4a0e-bd87-27c476581e86 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:22.115473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd278ef7-0f4d-4a0e-bd87-27c476581e86 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:22.115880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd278ef7-0f4d-4a0e-bd87-27c476581e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:22.143913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dd278ef7-0f4d-4a0e-bd87-27c476581e86 None None] MFA Rules not processed for user `9915b1158d3f45a6a143efd8eb95bdee`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:22.145129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dd278ef7-0f4d-4a0e-bd87-27c476581e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:22.184860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:22.219911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1047/2107] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:22.231325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315554ae-c05f-48d6-b7ff-87b38821b880 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:22.233282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-315554ae-c05f-48d6-b7ff-87b38821b880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:22.300562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315554ae-c05f-48d6-b7ff-87b38821b880 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:22.304475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e970631-ac0f-4053-9c4a-0f36684423bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:22.305122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e970631-ac0f-4053-9c4a-0f36684423bd service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:22.305399 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e970631-ac0f-4053-9c4a-0f36684423bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:22.305675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e970631-ac0f-4053-9c4a-0f36684423bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:22.366596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e970631-ac0f-4053-9c4a-0f36684423bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:22.368024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e970631-ac0f-4053-9c4a-0f36684423bd service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:22.377628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1061/2108] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:29.158596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd278ef7-0f4d-4a0e-bd87-27c476581e86 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:29.229143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd278ef7-0f4d-4a0e-bd87-27c476581e86 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:29.233761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:29.234771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:29.235114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:29.235607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] PATH_INFO: `/v3/users/162afb114d494d8ab1280b912bd9cc22` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:29.236860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=162afb114d494d8ab1280b912bd9cc22)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:29.238887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:29.288234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162afb114d494d8ab1280b912bd9cc22'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:29.288880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162afb114d494d8ab1280b912bd9cc22'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:29.301123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162afb114d494d8ab1280b912bd9cc22'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:29.301754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162afb114d494d8ab1280b912bd9cc22'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:29.308836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162afb114d494d8ab1280b912bd9cc22'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:29.327584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1048/2109] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:29 2022] DELETE /identity/v3/users/162afb114d494d8ab1280b912bd9cc22 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:29.332261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da6e68-9913-4560-871f-c6e899650062 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:29.332660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da6e68-9913-4560-871f-c6e899650062 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:29.332931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da6e68-9913-4560-871f-c6e899650062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:29.350648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d2da6e68-9913-4560-871f-c6e899650062 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:29.352254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d2da6e68-9913-4560-871f-c6e899650062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:29.414800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1062/2110] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:29.424818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:29.426698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:29.490585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39981f1-c314-4e53-937d-4adbeefee417 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:29.495144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba75721-671d-4fb7-a0e8-ac5c6922bbb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:29.495832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba75721-671d-4fb7-a0e8-ac5c6922bbb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:29.496113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba75721-671d-4fb7-a0e8-ac5c6922bbb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:29.496438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba75721-671d-4fb7-a0e8-ac5c6922bbb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:29.505413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:29.508390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba75721-671d-4fb7-a0e8-ac5c6922bbb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:29.510296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba75721-671d-4fb7-a0e8-ac5c6922bbb3 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:29.522443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1049/2111] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:29.820792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da6e68-9913-4560-871f-c6e899650062 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:29.822453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d2da6e68-9913-4560-871f-c6e899650062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:29.843150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da6e68-9913-4560-871f-c6e899650062 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:29.848656 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:29.849615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:29.850031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:29.850320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] PATH_INFO: `/v3/projects/7eccdb52f54f4a3db742b60e9bef9cff` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:29.851519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eccdb52f54f4a3db742b60e9bef9cff)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:29.853647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:29.893452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:29.912216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eccdb52f54f4a3db742b60e9bef9cff', 'request_id': 'req-6fcd8727-a601-4121-9d5d-3221665732dc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:29.922527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eccdb52f54f4a3db742b60e9bef9cff', 'request_id': 'req-6fcd8727-a601-4121-9d5d-3221665732dc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:29.929249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eccdb52f54f4a3db742b60e9bef9cff', 'request_id': 'req-6fcd8727-a601-4121-9d5d-3221665732dc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:29.941457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] The token cache is being invalidate because project 7eccdb52f54f4a3db742b60e9bef9cff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:29.941746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:29.942423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:29.943977 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1063/2112] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:29 2022] DELETE /identity/v3/projects/7eccdb52f54f4a3db742b60e9bef9cff => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:29.948195 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba75721-671d-4fb7-a0e8-ac5c6922bbb3 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:29.959198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba75721-671d-4fb7-a0e8-ac5c6922bbb3 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:30.019686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:30.020571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:30.020851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:30.021124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] PATH_INFO: `/v3/users/9915b1158d3f45a6a143efd8eb95bdee` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:30.022331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9915b1158d3f45a6a143efd8eb95bdee)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:30.024263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:30.074616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9915b1158d3f45a6a143efd8eb95bdee'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.075292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9915b1158d3f45a6a143efd8eb95bdee'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.095074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9915b1158d3f45a6a143efd8eb95bdee'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.097221 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9915b1158d3f45a6a143efd8eb95bdee'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.105540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9915b1158d3f45a6a143efd8eb95bdee'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.123200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1050/2113] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:29 2022] DELETE /identity/v3/users/9915b1158d3f45a6a143efd8eb95bdee => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:30.582171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:30.586967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:30.587994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:30.588724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:30.616438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd8727-a601-4121-9d5d-3221665732dc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:30.679979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51ecd-fd3b-4598-9219-af6e8b5ac28b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:30.684775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:30.685673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:30.685962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:30.686382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] PATH_INFO: `/v3/users/cc6583c280f54a7d92907820d5700623` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:30.687249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:30.687597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6583c280f54a7d92907820d5700623)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:30.690135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:30.690135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:30.690135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] PATH_INFO: `/v3/projects/184b39e6824f4942bfa255cd7324e873` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:30.690457 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:30.692204 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=184b39e6824f4942bfa255cd7324e873)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:30.694926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:30.711818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:30.765390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '184b39e6824f4942bfa255cd7324e873', 'request_id': 'req-bd0eb558-4521-4217-98de-09d22b131e7c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.768797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6583c280f54a7d92907820d5700623'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.769463 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6583c280f54a7d92907820d5700623'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.775369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '184b39e6824f4942bfa255cd7324e873', 'request_id': 'req-bd0eb558-4521-4217-98de-09d22b131e7c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.778542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6583c280f54a7d92907820d5700623'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.779165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6583c280f54a7d92907820d5700623'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.782225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '184b39e6824f4942bfa255cd7324e873', 'request_id': 'req-bd0eb558-4521-4217-98de-09d22b131e7c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.787513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2b4e2cda-4468-4a05-852c-7800bf5c9ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6583c280f54a7d92907820d5700623'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.796367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] The token cache is being invalidate because project 184b39e6824f4942bfa255cd7324e873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:30.796663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.797321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bd0eb558-4521-4217-98de-09d22b131e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:30.799167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1064/2114] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:30 2022] DELETE /identity/v3/projects/184b39e6824f4942bfa255cd7324e873 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:30.804582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7889d5a-aab7-4c50-8090-926959a73a9f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:30.804874 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7889d5a-aab7-4c50-8090-926959a73a9f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:30.805193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7889d5a-aab7-4c50-8090-926959a73a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:30.810154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1051/2115] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:30 2022] DELETE /identity/v3/users/cc6583c280f54a7d92907820d5700623 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:30.815413 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ef43-007c-4dc0-b3ba-af0f1bf64508 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:30.815711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ef43-007c-4dc0-b3ba-af0f1bf64508 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:30.816020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ef43-007c-4dc0-b3ba-af0f1bf64508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:30.825469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b7889d5a-aab7-4c50-8090-926959a73a9f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:30.826746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b7889d5a-aab7-4c50-8090-926959a73a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:30.835187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a307ef43-007c-4dc0-b3ba-af0f1bf64508 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:30.836431 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a307ef43-007c-4dc0-b3ba-af0f1bf64508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:30.868775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:30.891631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1065/2116] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:30.900550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1052/2117] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:30.900879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1779b40-548e-412e-8ef2-6262842a0f87 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:30.901181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1779b40-548e-412e-8ef2-6262842a0f87 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:30.901486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1779b40-548e-412e-8ef2-6262842a0f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:30.907334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307ef43-007c-4dc0-b3ba-af0f1bf64508 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:30.910929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a307ef43-007c-4dc0-b3ba-af0f1bf64508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:30.921851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d1779b40-548e-412e-8ef2-6262842a0f87 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:30.923114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d1779b40-548e-412e-8ef2-6262842a0f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:30.931381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1066/2118] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:30.935619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1779b40-548e-412e-8ef2-6262842a0f87 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:30.945251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1779b40-548e-412e-8ef2-6262842a0f87 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:30.949845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006427e-5077-4968-83a7-7274fd567420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:30.950541 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006427e-5077-4968-83a7-7274fd567420 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:30.950821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006427e-5077-4968-83a7-7274fd567420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:30.951115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006427e-5077-4968-83a7-7274fd567420 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:30.951857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006427e-5077-4968-83a7-7274fd567420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:30.952190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006427e-5077-4968-83a7-7274fd567420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:30.953822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006427e-5077-4968-83a7-7274fd567420 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:30.966419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1067/2119] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:05:30.970774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006427e-5077-4968-83a7-7274fd567420 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:30.972478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307ef43-007c-4dc0-b3ba-af0f1bf64508 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:30.976503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bb0f3-4893-4b6e-b1ab-340c829075e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:30.977684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bb0f3-4893-4b6e-b1ab-340c829075e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:30.977966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bb0f3-4893-4b6e-b1ab-340c829075e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:30.978274 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bb0f3-4893-4b6e-b1ab-340c829075e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:30.980244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006427e-5077-4968-83a7-7274fd567420 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:30.984944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c951f-a437-4bad-9e73-15a3eae0203c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:30.985891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c951f-a437-4bad-9e73-15a3eae0203c admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:30.986212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c951f-a437-4bad-9e73-15a3eae0203c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:30.986493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c951f-a437-4bad-9e73-15a3eae0203c admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:30.987218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c951f-a437-4bad-9e73-15a3eae0203c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:30.988861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bb0f3-4893-4b6e-b1ab-340c829075e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:30.989119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c951f-a437-4bad-9e73-15a3eae0203c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:30.990534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bb0f3-4893-4b6e-b1ab-340c829075e3 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.000853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1053/2120] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:31.015879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1068/2121] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:30 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:31.020775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bb0f3-4893-4b6e-b1ab-340c829075e3 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.030520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bb0f3-4893-4b6e-b1ab-340c829075e3 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.068347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:31.109943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b16d6e-3ea1-474d-ad15-8f1abde4a61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.110774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b16d6e-3ea1-474d-ad15-8f1abde4a61e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.111056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b16d6e-3ea1-474d-ad15-8f1abde4a61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.111331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b16d6e-3ea1-474d-ad15-8f1abde4a61e admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.112048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b16d6e-3ea1-474d-ad15-8f1abde4a61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.114136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b16d6e-3ea1-474d-ad15-8f1abde4a61e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.139621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1054/2122] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:31 2022] POST /identity/v3/users => generated 282 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:31.141125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c951f-a437-4bad-9e73-15a3eae0203c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.153309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c951f-a437-4bad-9e73-15a3eae0203c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.158559 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fad1b-7571-4a11-b834-12e2b20490a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.159369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fad1b-7571-4a11-b834-12e2b20490a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.159646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fad1b-7571-4a11-b834-12e2b20490a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.159783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fad1b-7571-4a11-b834-12e2b20490a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.160627 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320fad1b-7571-4a11-b834-12e2b20490a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.162163 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320fad1b-7571-4a11-b834-12e2b20490a1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.173922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1069/2123] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:05:31.177990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b16d6e-3ea1-474d-ad15-8f1abde4a61e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.188194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b16d6e-3ea1-474d-ad15-8f1abde4a61e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.193724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158bba4-5777-48ad-8d00-d61ef7490839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.194441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158bba4-5777-48ad-8d00-d61ef7490839 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.195085 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158bba4-5777-48ad-8d00-d61ef7490839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.195509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158bba4-5777-48ad-8d00-d61ef7490839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.196277 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158bba4-5777-48ad-8d00-d61ef7490839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.197905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158bba4-5777-48ad-8d00-d61ef7490839 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.209739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1055/2124] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:31.213162 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fad1b-7571-4a11-b834-12e2b20490a1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.223965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fad1b-7571-4a11-b834-12e2b20490a1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.229186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084102a-ca2b-425e-a6b4-0a1b0a5083de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.230111 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084102a-ca2b-425e-a6b4-0a1b0a5083de admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.230426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084102a-ca2b-425e-a6b4-0a1b0a5083de admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.230709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084102a-ca2b-425e-a6b4-0a1b0a5083de admin admin] PATH_INFO: `/v3/projects/a821e7003d274be79ee94c43df9d17c8/users/9c71707487fa4685994d1677ba69b127/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.246270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084102a-ca2b-425e-a6b4-0a1b0a5083de admin admin] RBAC: Authorizing `identity:create_grant(project_id=a821e7003d274be79ee94c43df9d17c8, user_id=9c71707487fa4685994d1677ba69b127, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.248978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084102a-ca2b-425e-a6b4-0a1b0a5083de admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.262499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1070/2125] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:31 2022] PUT /identity/v3/projects/a821e7003d274be79ee94c43df9d17c8/users/9c71707487fa4685994d1677ba69b127/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:31.277124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd5635-744e-4c64-8e72-99b4ce97d5ff None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.277419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd5635-744e-4c64-8e72-99b4ce97d5ff None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.277725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd5635-744e-4c64-8e72-99b4ce97d5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.291082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084102a-ca2b-425e-a6b4-0a1b0a5083de admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.293089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3084102a-ca2b-425e-a6b4-0a1b0a5083de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:31.307350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-17fd5635-744e-4c64-8e72-99b4ce97d5ff None None] MFA Rules not processed for user `9c71707487fa4685994d1677ba69b127`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:31.308955 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-17fd5635-744e-4c64-8e72-99b4ce97d5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:31.370513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084102a-ca2b-425e-a6b4-0a1b0a5083de admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.376468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.377356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.377642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.377924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] PATH_INFO: `/v3/projects/d27dc6069a79488890a6c6b1cda04d97` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.379157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27dc6069a79488890a6c6b1cda04d97)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.381608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.401583 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1056/2126] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:31.413169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd5635-744e-4c64-8e72-99b4ce97d5ff None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.414733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-17fd5635-744e-4c64-8e72-99b4ce97d5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:31.441052 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27dc6069a79488890a6c6b1cda04d97', 'request_id': 'req-6a15a210-bd64-4c95-8542-1e9df27a9332'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:31.451560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27dc6069a79488890a6c6b1cda04d97', 'request_id': 'req-6a15a210-bd64-4c95-8542-1e9df27a9332'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:31.458545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27dc6069a79488890a6c6b1cda04d97', 'request_id': 'req-6a15a210-bd64-4c95-8542-1e9df27a9332'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:31.467613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:31.472359 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] The token cache is being invalidate because project d27dc6069a79488890a6c6b1cda04d97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:31.473474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:31.474434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a15a210-bd64-4c95-8542-1e9df27a9332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:31.476360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1071/2127] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:31 2022] DELETE /identity/v3/projects/d27dc6069a79488890a6c6b1cda04d97 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:31.481370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22260de4-a39b-40a0-bd55-8b5392ffe22c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.481758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22260de4-a39b-40a0-bd55-8b5392ffe22c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.482155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22260de4-a39b-40a0-bd55-8b5392ffe22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.483518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd5635-744e-4c64-8e72-99b4ce97d5ff None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.505028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-22260de4-a39b-40a0-bd55-8b5392ffe22c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:31.506266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-22260de4-a39b-40a0-bd55-8b5392ffe22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:31.537286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acfa38-9d51-4b40-babf-3d91d692b2a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.537946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acfa38-9d51-4b40-babf-3d91d692b2a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.538263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acfa38-9d51-4b40-babf-3d91d692b2a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.538544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acfa38-9d51-4b40-babf-3d91d692b2a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.540370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-28acfa38-9d51-4b40-babf-3d91d692b2a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:31.558392 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:31.567168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1072/2128] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:31.575869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28094565-1238-4aa4-b461-c5884713e2d3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.576157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28094565-1238-4aa4-b461-c5884713e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.576745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28094565-1238-4aa4-b461-c5884713e2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.598523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-28094565-1238-4aa4-b461-c5884713e2d3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:31.600534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-28094565-1238-4aa4-b461-c5884713e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:31.605897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acfa38-9d51-4b40-babf-3d91d692b2a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.607395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acfa38-9d51-4b40-babf-3d91d692b2a7 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.610356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1073/2129] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:31.615109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28094565-1238-4aa4-b461-c5884713e2d3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.616889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1057/2130] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:31.624614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28094565-1238-4aa4-b461-c5884713e2d3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.629455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35a78e-a207-4174-bf71-de4a9bd30ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.630177 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35a78e-a207-4174-bf71-de4a9bd30ccb admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.630769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35a78e-a207-4174-bf71-de4a9bd30ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.630769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35a78e-a207-4174-bf71-de4a9bd30ccb admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.631423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35a78e-a207-4174-bf71-de4a9bd30ccb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:31.631737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35a78e-a207-4174-bf71-de4a9bd30ccb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.633428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35a78e-a207-4174-bf71-de4a9bd30ccb admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.648455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1074/2131] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:05:31.652822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acfa38-9d51-4b40-babf-3d91d692b2a7 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.669817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acfa38-9d51-4b40-babf-3d91d692b2a7 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.676633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac947b-bb0c-4bc7-9117-10afe2d9a01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.677314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac947b-bb0c-4bc7-9117-10afe2d9a01b admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.677596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac947b-bb0c-4bc7-9117-10afe2d9a01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.677867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac947b-bb0c-4bc7-9117-10afe2d9a01b admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.678612 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac947b-bb0c-4bc7-9117-10afe2d9a01b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.680527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac947b-bb0c-4bc7-9117-10afe2d9a01b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.715845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1058/2132] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:31 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:31.717280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35a78e-a207-4174-bf71-de4a9bd30ccb admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.732251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35a78e-a207-4174-bf71-de4a9bd30ccb admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.800928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17703b26-2391-4a39-a276-46e71b131226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.802517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703b26-2391-4a39-a276-46e71b131226 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.806473 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703b26-2391-4a39-a276-46e71b131226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.806875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703b26-2391-4a39-a276-46e71b131226 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.807629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17703b26-2391-4a39-a276-46e71b131226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.809978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17703b26-2391-4a39-a276-46e71b131226 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.833689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac947b-bb0c-4bc7-9117-10afe2d9a01b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.835599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dcac947b-bb0c-4bc7-9117-10afe2d9a01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:31.839091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1075/2133] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:31 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:31.843725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17703b26-2391-4a39-a276-46e71b131226 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.850891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac947b-bb0c-4bc7-9117-10afe2d9a01b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.855757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17703b26-2391-4a39-a276-46e71b131226 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.858319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.859676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.860227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.860541 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfc848-b5d2-4524-b30c-cd3b17976a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.861038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] PATH_INFO: `/v3/users/b43835cd6f224aeca7cb67e6c7ab31b5` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.861245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfc848-b5d2-4524-b30c-cd3b17976a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.861482 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfc848-b5d2-4524-b30c-cd3b17976a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.861757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfc848-b5d2-4524-b30c-cd3b17976a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.862530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfc848-b5d2-4524-b30c-cd3b17976a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.863331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b43835cd6f224aeca7cb67e6c7ab31b5)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.864144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfc848-b5d2-4524-b30c-cd3b17976a2d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.865896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.875441 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1076/2134] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:05:31.883244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfc848-b5d2-4524-b30c-cd3b17976a2d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.901796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfc848-b5d2-4524-b30c-cd3b17976a2d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.921989 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b43835cd6f224aeca7cb67e6c7ab31b5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:31.922862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b43835cd6f224aeca7cb67e6c7ab31b5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:31.933774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17f608-5fe1-4005-a675-1898e34c0d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.934923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17f608-5fe1-4005-a675-1898e34c0d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.935321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17f608-5fe1-4005-a675-1898e34c0d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.935638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17f608-5fe1-4005-a675-1898e34c0d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.936582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17f608-5fe1-4005-a675-1898e34c0d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:31.938003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b43835cd6f224aeca7cb67e6c7ab31b5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:31.938805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17f608-5fe1-4005-a675-1898e34c0d45 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:31.939798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b43835cd6f224aeca7cb67e6c7ab31b5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:31.950368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37801e32-319a-41c2-8fe4-b768ab8f6c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b43835cd6f224aeca7cb67e6c7ab31b5'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:31.955017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1077/2135] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:31.956334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17f608-5fe1-4005-a675-1898e34c0d45 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:31.958244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ec17f608-5fe1-4005-a675-1898e34c0d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:31.972441 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17f608-5fe1-4005-a675-1898e34c0d45 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:31.979479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6ffa9-1542-49ea-a009-0199f91cc841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:31.980532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6ffa9-1542-49ea-a009-0199f91cc841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.980822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6ffa9-1542-49ea-a009-0199f91cc841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.981108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6ffa9-1542-49ea-a009-0199f91cc841 admin admin] PATH_INFO: `/v3/projects/c52a8ced0af040e49f70d483e8ab1d8a/users/2928d6539a8148eb967761635aac33ec/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:31.987508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1059/2136] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:31 2022] DELETE /identity/v3/users/b43835cd6f224aeca7cb67e6c7ab31b5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:31.992055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608773f5-7106-4dd0-ba67-1a0436dc964f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:31.992355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608773f5-7106-4dd0-ba67-1a0436dc964f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:31.992632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608773f5-7106-4dd0-ba67-1a0436dc964f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:32.005270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6ffa9-1542-49ea-a009-0199f91cc841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52a8ced0af040e49f70d483e8ab1d8a, user_id=2928d6539a8148eb967761635aac33ec, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:32.008528 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6ffa9-1542-49ea-a009-0199f91cc841 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:32.016537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-608773f5-7106-4dd0-ba67-1a0436dc964f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:32.018118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-608773f5-7106-4dd0-ba67-1a0436dc964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:32.030155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1078/2137] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:31 2022] PUT /identity/v3/projects/c52a8ced0af040e49f70d483e8ab1d8a/users/2928d6539a8148eb967761635aac33ec/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:32.038367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e19447-a867-40b7-8185-cbcaccfd3bfb None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:32.038843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e19447-a867-40b7-8185-cbcaccfd3bfb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:32.039325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e19447-a867-40b7-8185-cbcaccfd3bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:32.070723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-25e19447-a867-40b7-8185-cbcaccfd3bfb None None] MFA Rules not processed for user `2928d6539a8148eb967761635aac33ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:32.072005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-25e19447-a867-40b7-8185-cbcaccfd3bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:32.085021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:32.096674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1060/2138] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:32.103558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608773f5-7106-4dd0-ba67-1a0436dc964f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:32.105359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-608773f5-7106-4dd0-ba67-1a0436dc964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:32.115375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:32.150300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1079/2139] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:32.161867 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e19447-a867-40b7-8185-cbcaccfd3bfb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:32.163332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-25e19447-a867-40b7-8185-cbcaccfd3bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:32.170433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608773f5-7106-4dd0-ba67-1a0436dc964f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:32.174520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844c0a8-62a5-4391-bed2-f7d967c9b899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:32.175181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844c0a8-62a5-4391-bed2-f7d967c9b899 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:32.175459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844c0a8-62a5-4391-bed2-f7d967c9b899 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:32.175745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844c0a8-62a5-4391-bed2-f7d967c9b899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:32.186761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844c0a8-62a5-4391-bed2-f7d967c9b899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:32.188614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844c0a8-62a5-4391-bed2-f7d967c9b899 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:32.199171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1061/2140] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:32.223503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e19447-a867-40b7-8185-cbcaccfd3bfb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:32.227570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3de59-4ecb-458c-8a77-ce32dc0804d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:32.228231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3de59-4ecb-458c-8a77-ce32dc0804d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:32.228513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3de59-4ecb-458c-8a77-ce32dc0804d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:32.228793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3de59-4ecb-458c-8a77-ce32dc0804d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:32.238123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3de59-4ecb-458c-8a77-ce32dc0804d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:32.239629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3de59-4ecb-458c-8a77-ce32dc0804d7 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:32.250578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1080/2141] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:32.492705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844c0a8-62a5-4391-bed2-f7d967c9b899 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:32.495532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f844c0a8-62a5-4391-bed2-f7d967c9b899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:32.532378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844c0a8-62a5-4391-bed2-f7d967c9b899 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:32.554800 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:32.555696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:32.555977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:32.556239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] PATH_INFO: `/v3/projects/86fba0c733be44b8beadefbe93da79c2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:32.558114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86fba0c733be44b8beadefbe93da79c2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:32.560061 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:32.645387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86fba0c733be44b8beadefbe93da79c2', 'request_id': 'req-f3eeaadf-0db2-4111-b873-809778cba623'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:32.654604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86fba0c733be44b8beadefbe93da79c2', 'request_id': 'req-f3eeaadf-0db2-4111-b873-809778cba623'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:32.661651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86fba0c733be44b8beadefbe93da79c2', 'request_id': 'req-f3eeaadf-0db2-4111-b873-809778cba623'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:32.674323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] The token cache is being invalidate because project 86fba0c733be44b8beadefbe93da79c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:32.674615 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:32.675302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f3eeaadf-0db2-4111-b873-809778cba623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:32.677075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1062/2142] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:32 2022] DELETE /identity/v3/projects/86fba0c733be44b8beadefbe93da79c2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:32.681815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2927281-5143-49d8-a7ef-252b70bbfeab None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:32.682392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2927281-5143-49d8-a7ef-252b70bbfeab None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:32.682392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2927281-5143-49d8-a7ef-252b70bbfeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:32.701729 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f2927281-5143-49d8-a7ef-252b70bbfeab None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:32.703009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f2927281-5143-49d8-a7ef-252b70bbfeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:32.772609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1081/2143] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:32.780565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72466ec3-eaf3-41c4-b35a-ca0ac5abe9fe None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:32.780836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72466ec3-eaf3-41c4-b35a-ca0ac5abe9fe None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:32.781112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72466ec3-eaf3-41c4-b35a-ca0ac5abe9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:32.800353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-72466ec3-eaf3-41c4-b35a-ca0ac5abe9fe None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:32.801567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-72466ec3-eaf3-41c4-b35a-ca0ac5abe9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:32.809907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1063/2144] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:32.814031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2927281-5143-49d8-a7ef-252b70bbfeab None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:32.824361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2927281-5143-49d8-a7ef-252b70bbfeab None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:32.829128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23e7c1-60ce-4dc0-ac58-23faa891b58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:32.829796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23e7c1-60ce-4dc0-ac58-23faa891b58e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:32.830230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23e7c1-60ce-4dc0-ac58-23faa891b58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:32.830507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23e7c1-60ce-4dc0-ac58-23faa891b58e admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:32.831235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23e7c1-60ce-4dc0-ac58-23faa891b58e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:32.831540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23e7c1-60ce-4dc0-ac58-23faa891b58e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:32.833701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23e7c1-60ce-4dc0-ac58-23faa891b58e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:32.847123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1082/2145] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:05:32.851940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72466ec3-eaf3-41c4-b35a-ca0ac5abe9fe None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:32.861789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72466ec3-eaf3-41c4-b35a-ca0ac5abe9fe None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:32.866860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b66f25-4474-431a-840b-6c8127767bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:32.867617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b66f25-4474-431a-840b-6c8127767bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:32.868543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b66f25-4474-431a-840b-6c8127767bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:32.868833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b66f25-4474-431a-840b-6c8127767bc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:32.869557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b66f25-4474-431a-840b-6c8127767bc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:32.871551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b66f25-4474-431a-840b-6c8127767bc8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:32.902829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1064/2146] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:32 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:32.904340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23e7c1-60ce-4dc0-ac58-23faa891b58e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:32.915385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23e7c1-60ce-4dc0-ac58-23faa891b58e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:32.959232 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:32.977173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a35c9-82fa-4179-8b6e-f4b23524ac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:32.977873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a35c9-82fa-4179-8b6e-f4b23524ac04 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:32.978185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a35c9-82fa-4179-8b6e-f4b23524ac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:32.978476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a35c9-82fa-4179-8b6e-f4b23524ac04 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:32.979149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a35c9-82fa-4179-8b6e-f4b23524ac04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:32.981053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a35c9-82fa-4179-8b6e-f4b23524ac04 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.008506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1083/2147] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:32 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:33.009878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b66f25-4474-431a-840b-6c8127767bc8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.020365 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b66f25-4474-431a-840b-6c8127767bc8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.025176 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf86e81-f583-4280-9566-94d23f061b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.025833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf86e81-f583-4280-9566-94d23f061b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.026184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf86e81-f583-4280-9566-94d23f061b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.026460 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf86e81-f583-4280-9566-94d23f061b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.027258 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf86e81-f583-4280-9566-94d23f061b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.028940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf86e81-f583-4280-9566-94d23f061b1f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.039961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1065/2148] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:33.043911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a35c9-82fa-4179-8b6e-f4b23524ac04 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.054282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a35c9-82fa-4179-8b6e-f4b23524ac04 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.059087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bbfc7-6425-457f-8a47-82923ba4df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.059738 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bbfc7-6425-457f-8a47-82923ba4df63 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.060015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bbfc7-6425-457f-8a47-82923ba4df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.060288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bbfc7-6425-457f-8a47-82923ba4df63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.061009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bbfc7-6425-457f-8a47-82923ba4df63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.062645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bbfc7-6425-457f-8a47-82923ba4df63 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.073754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1084/2149] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:05:33.077569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf86e81-f583-4280-9566-94d23f061b1f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.088700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf86e81-f583-4280-9566-94d23f061b1f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.093726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269841b7-72b5-4409-9058-7ee47d13a5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.095865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269841b7-72b5-4409-9058-7ee47d13a5d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.096120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269841b7-72b5-4409-9058-7ee47d13a5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.096517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269841b7-72b5-4409-9058-7ee47d13a5d8 admin admin] PATH_INFO: `/v3/projects/475fec527f064cfabe7733a58d9d9a34/users/f227d722919d4d4e8afd14491d95eb55/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.112652 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269841b7-72b5-4409-9058-7ee47d13a5d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=475fec527f064cfabe7733a58d9d9a34, user_id=f227d722919d4d4e8afd14491d95eb55, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.115373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269841b7-72b5-4409-9058-7ee47d13a5d8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.131941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1066/2150] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:33 2022] PUT /identity/v3/projects/475fec527f064cfabe7733a58d9d9a34/users/f227d722919d4d4e8afd14491d95eb55/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:33.143250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d0a8f-2d1b-4107-be54-31910940c5f8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.143529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d0a8f-2d1b-4107-be54-31910940c5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.143806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d0a8f-2d1b-4107-be54-31910940c5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.170856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-483d0a8f-2d1b-4107-be54-31910940c5f8 None None] MFA Rules not processed for user `f227d722919d4d4e8afd14491d95eb55`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:33.172078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-483d0a8f-2d1b-4107-be54-31910940c5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:33.238663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1085/2151] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:33.243303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269841b7-72b5-4409-9058-7ee47d13a5d8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.278976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:33.312878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269841b7-72b5-4409-9058-7ee47d13a5d8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.317618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e7fff-3374-46ba-ba3a-be6e0a4ace61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.318350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e7fff-3374-46ba-ba3a-be6e0a4ace61 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.318633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e7fff-3374-46ba-ba3a-be6e0a4ace61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.318905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e7fff-3374-46ba-ba3a-be6e0a4ace61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.319964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e7fff-3374-46ba-ba3a-be6e0a4ace61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.321900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e7fff-3374-46ba-ba3a-be6e0a4ace61 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.348620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1067/2152] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:33 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:33.353050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d0a8f-2d1b-4107-be54-31910940c5f8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.363953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d0a8f-2d1b-4107-be54-31910940c5f8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.424107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d0848-8fd7-4c50-9f41-9349eab07e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.424779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d0848-8fd7-4c50-9f41-9349eab07e7c admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.425061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d0848-8fd7-4c50-9f41-9349eab07e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.425330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d0848-8fd7-4c50-9f41-9349eab07e7c admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.426010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d0848-8fd7-4c50-9f41-9349eab07e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.427969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d0848-8fd7-4c50-9f41-9349eab07e7c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.450658 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1086/2153] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:33 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:33.454677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e7fff-3374-46ba-ba3a-be6e0a4ace61 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.465672 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e7fff-3374-46ba-ba3a-be6e0a4ace61 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.470908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d58ab8-2fd4-41cb-a243-6470a0a980bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.471594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d58ab8-2fd4-41cb-a243-6470a0a980bc admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.471848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d58ab8-2fd4-41cb-a243-6470a0a980bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.472156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d58ab8-2fd4-41cb-a243-6470a0a980bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.472897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d58ab8-2fd4-41cb-a243-6470a0a980bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.474603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d58ab8-2fd4-41cb-a243-6470a0a980bc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.486161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1068/2154] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:33.490367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d0848-8fd7-4c50-9f41-9349eab07e7c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.501075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d0848-8fd7-4c50-9f41-9349eab07e7c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.507127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec0e0d-49d4-4729-b2bf-2f3db968aca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.507803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec0e0d-49d4-4729-b2bf-2f3db968aca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.508083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec0e0d-49d4-4729-b2bf-2f3db968aca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.508357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec0e0d-49d4-4729-b2bf-2f3db968aca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.509094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec0e0d-49d4-4729-b2bf-2f3db968aca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.510772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec0e0d-49d4-4729-b2bf-2f3db968aca1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.525407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1087/2155] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:33.526711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d58ab8-2fd4-41cb-a243-6470a0a980bc admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.537390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d58ab8-2fd4-41cb-a243-6470a0a980bc admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.542571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b6bd0-3a06-41b6-849e-251ae92d2141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.543493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b6bd0-3a06-41b6-849e-251ae92d2141 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.543771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b6bd0-3a06-41b6-849e-251ae92d2141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.544046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b6bd0-3a06-41b6-849e-251ae92d2141 admin admin] PATH_INFO: `/v3/projects/0fd04ca3ab72418abfec2648373140cc/users/8b2cc7e700374279af9a372b83358124/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.562026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b6bd0-3a06-41b6-849e-251ae92d2141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd04ca3ab72418abfec2648373140cc, user_id=8b2cc7e700374279af9a372b83358124, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.565029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b6bd0-3a06-41b6-849e-251ae92d2141 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.579927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1069/2156] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:33 2022] PUT /identity/v3/projects/0fd04ca3ab72418abfec2648373140cc/users/8b2cc7e700374279af9a372b83358124/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:33.589646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7198ac-76db-48cc-8df7-c7ce1a28e7e1 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.589934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7198ac-76db-48cc-8df7-c7ce1a28e7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.590301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7198ac-76db-48cc-8df7-c7ce1a28e7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.620343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bb7198ac-76db-48cc-8df7-c7ce1a28e7e1 None None] MFA Rules not processed for user `8b2cc7e700374279af9a372b83358124`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:33.621670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bb7198ac-76db-48cc-8df7-c7ce1a28e7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:33.645206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:33.706566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1088/2157] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:33.711672 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b6bd0-3a06-41b6-849e-251ae92d2141 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.779861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b6bd0-3a06-41b6-849e-251ae92d2141 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.784917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01211cd4-2e88-4f00-a1e9-3b69041d11a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.785656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211cd4-2e88-4f00-a1e9-3b69041d11a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.785872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211cd4-2e88-4f00-a1e9-3b69041d11a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.786339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01211cd4-2e88-4f00-a1e9-3b69041d11a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.787051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01211cd4-2e88-4f00-a1e9-3b69041d11a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.788965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01211cd4-2e88-4f00-a1e9-3b69041d11a0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.821133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1070/2158] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:33 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:33.823016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7198ac-76db-48cc-8df7-c7ce1a28e7e1 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.834496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7198ac-76db-48cc-8df7-c7ce1a28e7e1 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.897254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5870c4d-06b6-4eb4-a858-72a0864f40b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.898310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5870c4d-06b6-4eb4-a858-72a0864f40b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.898715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5870c4d-06b6-4eb4-a858-72a0864f40b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.898998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5870c4d-06b6-4eb4-a858-72a0864f40b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.899728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5870c4d-06b6-4eb4-a858-72a0864f40b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.901680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5870c4d-06b6-4eb4-a858-72a0864f40b0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.929966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1089/2159] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:33 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:33.934227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01211cd4-2e88-4f00-a1e9-3b69041d11a0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.948208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01211cd4-2e88-4f00-a1e9-3b69041d11a0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.953690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935c0b8-351a-43c3-b2a9-b5343437ac28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.955068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c0b8-351a-43c3-b2a9-b5343437ac28 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.955816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c0b8-351a-43c3-b2a9-b5343437ac28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.955816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c0b8-351a-43c3-b2a9-b5343437ac28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:33.956365 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935c0b8-351a-43c3-b2a9-b5343437ac28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:33.957998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935c0b8-351a-43c3-b2a9-b5343437ac28 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:33.969398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1071/2160] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:33.973736 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5870c4d-06b6-4eb4-a858-72a0864f40b0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:33.985756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5870c4d-06b6-4eb4-a858-72a0864f40b0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:33.991479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee0b2-3b7b-4d09-b879-86b0644455ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:33.992388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee0b2-3b7b-4d09-b879-86b0644455ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:33.992733 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee0b2-3b7b-4d09-b879-86b0644455ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:33.993438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ee0b2-3b7b-4d09-b879-86b0644455ef admin admin] PATH_INFO: `/v3/projects/1ede1ea9d8f146399497d7028cc62962/users/cc3e2a9928674c84bd4ff5daa2dcf7f3/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:34.026113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ee0b2-3b7b-4d09-b879-86b0644455ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ede1ea9d8f146399497d7028cc62962, user_id=cc3e2a9928674c84bd4ff5daa2dcf7f3, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:34.029876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ee0b2-3b7b-4d09-b879-86b0644455ef admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:34.050067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1090/2161] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:33 2022] PUT /identity/v3/projects/1ede1ea9d8f146399497d7028cc62962/users/cc3e2a9928674c84bd4ff5daa2dcf7f3/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:34.061691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935c0b8-351a-43c3-b2a9-b5343437ac28 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:34.077967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935c0b8-351a-43c3-b2a9-b5343437ac28 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:34.164763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:34.178293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b878cd-4753-4a44-9c28-1dad00b04bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:34.179025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b878cd-4753-4a44-9c28-1dad00b04bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:34.179276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b878cd-4753-4a44-9c28-1dad00b04bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:34.179590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b878cd-4753-4a44-9c28-1dad00b04bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:34.180334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b878cd-4753-4a44-9c28-1dad00b04bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:34.181968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b878cd-4753-4a44-9c28-1dad00b04bfe admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:34.193383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1072/2162] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:34.197825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee0b2-3b7b-4d09-b879-86b0644455ef admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:34.212224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ee0b2-3b7b-4d09-b879-86b0644455ef admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:34.217422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d017ea-0621-463b-9ede-054f47af8b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:34.218439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d017ea-0621-463b-9ede-054f47af8b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:34.218725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d017ea-0621-463b-9ede-054f47af8b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:34.219004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d017ea-0621-463b-9ede-054f47af8b3d admin admin] PATH_INFO: `/v3/projects/1ede1ea9d8f146399497d7028cc62962/users/cc3e2a9928674c84bd4ff5daa2dcf7f3/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:34.221972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d017ea-0621-463b-9ede-054f47af8b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ede1ea9d8f146399497d7028cc62962, user_id=cc3e2a9928674c84bd4ff5daa2dcf7f3, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:34.232960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d017ea-0621-463b-9ede-054f47af8b3d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:34.255380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1091/2163] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:34 2022] PUT /identity/v3/projects/1ede1ea9d8f146399497d7028cc62962/users/cc3e2a9928674c84bd4ff5daa2dcf7f3/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:34.267035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72088d8a-3d5b-4e27-80ae-c177123293e1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:34.267376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72088d8a-3d5b-4e27-80ae-c177123293e1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:34.267663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72088d8a-3d5b-4e27-80ae-c177123293e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:34.303740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-72088d8a-3d5b-4e27-80ae-c177123293e1 None None] MFA Rules not processed for user `cc3e2a9928674c84bd4ff5daa2dcf7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:34.305967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-72088d8a-3d5b-4e27-80ae-c177123293e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:34.420819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1073/2164] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:34.578905 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d017ea-0621-463b-9ede-054f47af8b3d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:34.579013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-87d017ea-0621-463b-9ede-054f47af8b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:34.654660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d017ea-0621-463b-9ede-054f47af8b3d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:34.659068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf36a77-6e9a-4e1a-90bd-3b09f8c70e0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:34.660483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf36a77-6e9a-4e1a-90bd-3b09f8c70e0b service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:34.661178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf36a77-6e9a-4e1a-90bd-3b09f8c70e0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:34.661867 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf36a77-6e9a-4e1a-90bd-3b09f8c70e0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:34.700644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:34.748951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf36a77-6e9a-4e1a-90bd-3b09f8c70e0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:34.750495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf36a77-6e9a-4e1a-90bd-3b09f8c70e0b service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:34.762984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1092/2165] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:37.361311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72088d8a-3d5b-4e27-80ae-c177123293e1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:37.371873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72088d8a-3d5b-4e27-80ae-c177123293e1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:37.375898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5418e2aa-b6ab-4f32-b52f-53dfacb06eb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:37.376562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418e2aa-b6ab-4f32-b52f-53dfacb06eb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:37.376849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418e2aa-b6ab-4f32-b52f-53dfacb06eb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:37.377140 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5418e2aa-b6ab-4f32-b52f-53dfacb06eb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:37.438851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418e2aa-b6ab-4f32-b52f-53dfacb06eb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:37.440403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5418e2aa-b6ab-4f32-b52f-53dfacb06eb3 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:37.450322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1074/2166] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:37.566248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf36a77-6e9a-4e1a-90bd-3b09f8c70e0b service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:37.570068 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5bf36a77-6e9a-4e1a-90bd-3b09f8c70e0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:37.665313 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf36a77-6e9a-4e1a-90bd-3b09f8c70e0b service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:37.674357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:37.675289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:37.675735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:37.676182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] PATH_INFO: `/v3/users/2928d6539a8148eb967761635aac33ec` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:37.678100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2928d6539a8148eb967761635aac33ec)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:37.681156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:37.750221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2928d6539a8148eb967761635aac33ec'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:37.758573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2928d6539a8148eb967761635aac33ec'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:37.759551 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2928d6539a8148eb967761635aac33ec'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:37.760779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2928d6539a8148eb967761635aac33ec'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:37.776563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2928d6539a8148eb967761635aac33ec'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:37.792796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1093/2167] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:37 2022] DELETE /identity/v3/users/2928d6539a8148eb967761635aac33ec => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:37.797573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a5fa2-1335-4cfe-9fab-c22b1e5ee6ec None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:37.798044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a5fa2-1335-4cfe-9fab-c22b1e5ee6ec None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:37.798456 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a5fa2-1335-4cfe-9fab-c22b1e5ee6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:37.816833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-879a5fa2-1335-4cfe-9fab-c22b1e5ee6ec None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:37.818021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-879a5fa2-1335-4cfe-9fab-c22b1e5ee6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:37.895382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1075/2168] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:37.924491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:37.926031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:37.991475 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:37.999120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a0219-3f65-487b-aba0-d9d9d8a3f07e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:38.008474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380e22b-93d9-423b-8a4b-c04b432554fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:38.009877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380e22b-93d9-423b-8a4b-c04b432554fa service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:38.010285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380e22b-93d9-423b-8a4b-c04b432554fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:38.010629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380e22b-93d9-423b-8a4b-c04b432554fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:38.025839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380e22b-93d9-423b-8a4b-c04b432554fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:38.027548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380e22b-93d9-423b-8a4b-c04b432554fa service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:38.040288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1094/2169] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:38.153896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a5fa2-1335-4cfe-9fab-c22b1e5ee6ec None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:38.155452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-879a5fa2-1335-4cfe-9fab-c22b1e5ee6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:38.170609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:38.243873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a5fa2-1335-4cfe-9fab-c22b1e5ee6ec None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:38.247729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b2917-91b0-40cd-a6b1-18925ec6ca1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:38.248402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b2917-91b0-40cd-a6b1-18925ec6ca1a service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:38.248695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b2917-91b0-40cd-a6b1-18925ec6ca1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:38.248969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b2917-91b0-40cd-a6b1-18925ec6ca1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:38.251187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-159b2917-91b0-40cd-a6b1-18925ec6ca1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:38.335587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b2917-91b0-40cd-a6b1-18925ec6ca1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:38.338359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b2917-91b0-40cd-a6b1-18925ec6ca1a service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:38.361932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1076/2170] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:38.595648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380e22b-93d9-423b-8a4b-c04b432554fa service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:38.597245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3380e22b-93d9-423b-8a4b-c04b432554fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:38.612851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380e22b-93d9-423b-8a4b-c04b432554fa service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:38.619225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:38.620128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:38.620419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:38.620700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] PATH_INFO: `/v3/projects/c52a8ced0af040e49f70d483e8ab1d8a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:38.622034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c52a8ced0af040e49f70d483e8ab1d8a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:38.624205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:38.675885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c52a8ced0af040e49f70d483e8ab1d8a', 'request_id': 'req-015a906f-ba88-4444-b01e-dc947a100cd7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:38.685031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c52a8ced0af040e49f70d483e8ab1d8a', 'request_id': 'req-015a906f-ba88-4444-b01e-dc947a100cd7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:38.691246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c52a8ced0af040e49f70d483e8ab1d8a', 'request_id': 'req-015a906f-ba88-4444-b01e-dc947a100cd7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:38.703348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] The token cache is being invalidate because project c52a8ced0af040e49f70d483e8ab1d8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:38.703694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:38.704420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-015a906f-ba88-4444-b01e-dc947a100cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:38.706330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1095/2171] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:38 2022] DELETE /identity/v3/projects/c52a8ced0af040e49f70d483e8ab1d8a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:38.712359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aaa39d-3f86-47d4-9671-562c6551d5c1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:38.712649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aaa39d-3f86-47d4-9671-562c6551d5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:38.712961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aaa39d-3f86-47d4-9671-562c6551d5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:38.732408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-80aaa39d-3f86-47d4-9671-562c6551d5c1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:38.733617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-80aaa39d-3f86-47d4-9671-562c6551d5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:38.813748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1077/2172] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:38.824619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57110e-883a-48f8-bf7e-f60359702f6c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:38.824904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57110e-883a-48f8-bf7e-f60359702f6c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:38.825194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57110e-883a-48f8-bf7e-f60359702f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:38.844990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2b57110e-883a-48f8-bf7e-f60359702f6c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:38.846229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2b57110e-883a-48f8-bf7e-f60359702f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:38.856055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1096/2173] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:38.860519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aaa39d-3f86-47d4-9671-562c6551d5c1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:38.870795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aaa39d-3f86-47d4-9671-562c6551d5c1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:38.878271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ef497-22ec-4d9c-9210-c79f3b5b4758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:38.878958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef497-22ec-4d9c-9210-c79f3b5b4758 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:38.879243 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef497-22ec-4d9c-9210-c79f3b5b4758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:38.879948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef497-22ec-4d9c-9210-c79f3b5b4758 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:38.880555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ef497-22ec-4d9c-9210-c79f3b5b4758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:38.880865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ef497-22ec-4d9c-9210-c79f3b5b4758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:38.883785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ef497-22ec-4d9c-9210-c79f3b5b4758 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:38.915841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1078/2174] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:05:38.919583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57110e-883a-48f8-bf7e-f60359702f6c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:38.930784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57110e-883a-48f8-bf7e-f60359702f6c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:38.936303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb6a0e-f623-4723-916c-24913a5678ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:38.937198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb6a0e-f623-4723-916c-24913a5678ea admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:38.937530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb6a0e-f623-4723-916c-24913a5678ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:38.937812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb6a0e-f623-4723-916c-24913a5678ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:38.938564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb6a0e-f623-4723-916c-24913a5678ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:38.940484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb6a0e-f623-4723-916c-24913a5678ea admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:38.972855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1097/2175] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:38 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:38.978556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ef497-22ec-4d9c-9210-c79f3b5b4758 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:38.991157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ef497-22ec-4d9c-9210-c79f3b5b4758 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:39.051258 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:39.062637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871ded3-363f-4bce-bac0-aa0d0ba74887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:39.063373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ded3-363f-4bce-bac0-aa0d0ba74887 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:39.063655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ded3-363f-4bce-bac0-aa0d0ba74887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:39.063969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ded3-363f-4bce-bac0-aa0d0ba74887 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:39.064668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871ded3-363f-4bce-bac0-aa0d0ba74887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:39.066693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871ded3-363f-4bce-bac0-aa0d0ba74887 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:39.095945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1079/2176] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:38 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:39.100365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb6a0e-f623-4723-916c-24913a5678ea admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:39.111172 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb6a0e-f623-4723-916c-24913a5678ea admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:39.116120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5114799-a6ee-48a7-90ee-ed107776a902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:39.116822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5114799-a6ee-48a7-90ee-ed107776a902 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:39.117109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5114799-a6ee-48a7-90ee-ed107776a902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:39.117365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5114799-a6ee-48a7-90ee-ed107776a902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:39.118126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5114799-a6ee-48a7-90ee-ed107776a902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:39.119814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5114799-a6ee-48a7-90ee-ed107776a902 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:39.142039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1098/2177] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:39.145079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871ded3-363f-4bce-bac0-aa0d0ba74887 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:39.159450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871ded3-363f-4bce-bac0-aa0d0ba74887 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:39.171209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd8c9-cc43-47d0-9dea-deeae3b21867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:39.172027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd8c9-cc43-47d0-9dea-deeae3b21867 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:39.172343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd8c9-cc43-47d0-9dea-deeae3b21867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:39.172768 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd8c9-cc43-47d0-9dea-deeae3b21867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:39.173596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd8c9-cc43-47d0-9dea-deeae3b21867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:39.175549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd8c9-cc43-47d0-9dea-deeae3b21867 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:39.201018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1080/2178] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:39.203148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5114799-a6ee-48a7-90ee-ed107776a902 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:39.216668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5114799-a6ee-48a7-90ee-ed107776a902 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:39.222416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5a3fb-e644-47b4-9f05-2f08d15ac876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:39.223578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5a3fb-e644-47b4-9f05-2f08d15ac876 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:39.224591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5a3fb-e644-47b4-9f05-2f08d15ac876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:39.224933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5a3fb-e644-47b4-9f05-2f08d15ac876 admin admin] PATH_INFO: `/v3/projects/d47c143cf33a41799202d5f7d90a66b5/users/a96fa98fa00c4de29a9bed22ed9e597b/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:39.242973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5a3fb-e644-47b4-9f05-2f08d15ac876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47c143cf33a41799202d5f7d90a66b5, user_id=a96fa98fa00c4de29a9bed22ed9e597b, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:39.245940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5a3fb-e644-47b4-9f05-2f08d15ac876 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:39.264462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1099/2179] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:39 2022] PUT /identity/v3/projects/d47c143cf33a41799202d5f7d90a66b5/users/a96fa98fa00c4de29a9bed22ed9e597b/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:39.269494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2940c-834f-4dd4-9d24-4ed463133296 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:39.269833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2940c-834f-4dd4-9d24-4ed463133296 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:39.270268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2940c-834f-4dd4-9d24-4ed463133296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:39.289984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-38a2940c-834f-4dd4-9d24-4ed463133296 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:39.291492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-38a2940c-834f-4dd4-9d24-4ed463133296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:39.368909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1081/2180] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:39.379300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5a3fb-e644-47b4-9f05-2f08d15ac876 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:39.381178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d9e5a3fb-e644-47b4-9f05-2f08d15ac876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:39.492214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5a3fb-e644-47b4-9f05-2f08d15ac876 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:39.500836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea77fb-25f0-4858-8c58-f664988a6726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:39.503499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea77fb-25f0-4858-8c58-f664988a6726 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:39.504012 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea77fb-25f0-4858-8c58-f664988a6726 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:39.504468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea77fb-25f0-4858-8c58-f664988a6726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:39.527369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea77fb-25f0-4858-8c58-f664988a6726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:39.529083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea77fb-25f0-4858-8c58-f664988a6726 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:39.569764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1100/2181] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:40.536470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ae544-74ee-4bb9-905d-9ef456554687 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:40.536991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ae544-74ee-4bb9-905d-9ef456554687 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:40.537189 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ae544-74ee-4bb9-905d-9ef456554687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:40.566018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dc2ae544-74ee-4bb9-905d-9ef456554687 None None] MFA Rules not processed for user `a96fa98fa00c4de29a9bed22ed9e597b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:40.567329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dc2ae544-74ee-4bb9-905d-9ef456554687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:40.658650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1082/2182] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:40.666391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea77fb-25f0-4858-8c58-f664988a6726 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:40.668189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bfea77fb-25f0-4858-8c58-f664988a6726 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:40.687608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:40.753102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea77fb-25f0-4858-8c58-f664988a6726 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:40.759154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ac807-0ce4-4e78-ae49-0349014afa8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:40.760312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ac807-0ce4-4e78-ae49-0349014afa8f service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:40.760732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ac807-0ce4-4e78-ae49-0349014afa8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:40.761540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ac807-0ce4-4e78-ae49-0349014afa8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:40.774996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ac807-0ce4-4e78-ae49-0349014afa8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:40.777499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ac807-0ce4-4e78-ae49-0349014afa8f service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:40.797218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1101/2183] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:42.084491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ae544-74ee-4bb9-905d-9ef456554687 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:42.086060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dc2ae544-74ee-4bb9-905d-9ef456554687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:42.100545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ae544-74ee-4bb9-905d-9ef456554687 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:42.107155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:42.108140 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:42.108614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:42.109250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] PATH_INFO: `/v3/users/14d3e513d8ce4efc87a359e04a77f05b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:42.110598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d3e513d8ce4efc87a359e04a77f05b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:42.112761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:42.176932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d3e513d8ce4efc87a359e04a77f05b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:42.177622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d3e513d8ce4efc87a359e04a77f05b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:42.187586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d3e513d8ce4efc87a359e04a77f05b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:42.188250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d3e513d8ce4efc87a359e04a77f05b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:42.189905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:42.199789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1e5ed55c-b7a8-4dc8-8a41-0c06089d7a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d3e513d8ce4efc87a359e04a77f05b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:42.228135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1083/2184] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:42 2022] DELETE /identity/v3/users/14d3e513d8ce4efc87a359e04a77f05b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:42.465726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ac807-0ce4-4e78-ae49-0349014afa8f service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:42.483224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ac807-0ce4-4e78-ae49-0349014afa8f service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:42.543801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:42.544679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:42.544960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:42.545237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] PATH_INFO: `/v3/projects/1e27652a0a2f4421aed586e23a86d0ae` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:42.546474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e27652a0a2f4421aed586e23a86d0ae)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:42.548391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:42.596909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e27652a0a2f4421aed586e23a86d0ae', 'request_id': 'req-bfd206e8-99e4-4f43-b6fc-417cc3156d05'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:42.607328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e27652a0a2f4421aed586e23a86d0ae', 'request_id': 'req-bfd206e8-99e4-4f43-b6fc-417cc3156d05'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:42.613542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e27652a0a2f4421aed586e23a86d0ae', 'request_id': 'req-bfd206e8-99e4-4f43-b6fc-417cc3156d05'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:42.625358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] The token cache is being invalidate because project 1e27652a0a2f4421aed586e23a86d0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:42.625641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:42.626356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bfd206e8-99e4-4f43-b6fc-417cc3156d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:42.627932 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1102/2185] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:42 2022] DELETE /identity/v3/projects/1e27652a0a2f4421aed586e23a86d0ae => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:42.633252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b226f8e-575e-4d9a-9c95-30dbe4fb1006 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:42.633546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b226f8e-575e-4d9a-9c95-30dbe4fb1006 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:42.633865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b226f8e-575e-4d9a-9c95-30dbe4fb1006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:42.652556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9b226f8e-575e-4d9a-9c95-30dbe4fb1006 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:42.653764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9b226f8e-575e-4d9a-9c95-30dbe4fb1006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:42.722701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1084/2186] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:42.731605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95444ec9-5c7b-4bdf-a90f-aa12ad65341a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:42.731889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95444ec9-5c7b-4bdf-a90f-aa12ad65341a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:42.732166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95444ec9-5c7b-4bdf-a90f-aa12ad65341a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:42.750371 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-95444ec9-5c7b-4bdf-a90f-aa12ad65341a None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:42.751487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-95444ec9-5c7b-4bdf-a90f-aa12ad65341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:42.760320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1103/2187] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:42.764521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b226f8e-575e-4d9a-9c95-30dbe4fb1006 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:42.774357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b226f8e-575e-4d9a-9c95-30dbe4fb1006 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:42.779317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07496531-db8d-4350-a78a-edbbff09e18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:42.779981 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07496531-db8d-4350-a78a-edbbff09e18d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:42.780256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07496531-db8d-4350-a78a-edbbff09e18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:42.780530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07496531-db8d-4350-a78a-edbbff09e18d admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:42.781262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07496531-db8d-4350-a78a-edbbff09e18d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:42.781571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07496531-db8d-4350-a78a-edbbff09e18d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:42.783227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07496531-db8d-4350-a78a-edbbff09e18d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:42.795483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1085/2188] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:05:42.799967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95444ec9-5c7b-4bdf-a90f-aa12ad65341a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:42.810046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95444ec9-5c7b-4bdf-a90f-aa12ad65341a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:42.815029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085838ea-cc5a-4e86-ad16-7ca7a3e1e667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:42.815733 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085838ea-cc5a-4e86-ad16-7ca7a3e1e667 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:42.816020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085838ea-cc5a-4e86-ad16-7ca7a3e1e667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:42.816294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085838ea-cc5a-4e86-ad16-7ca7a3e1e667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:42.817242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085838ea-cc5a-4e86-ad16-7ca7a3e1e667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:42.819216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085838ea-cc5a-4e86-ad16-7ca7a3e1e667 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:42.846431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1104/2189] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:42 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:42.851216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07496531-db8d-4350-a78a-edbbff09e18d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:42.861378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07496531-db8d-4350-a78a-edbbff09e18d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:42.929178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dae44-5ce0-477c-ac63-fb3d2725a0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:42.929865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dae44-5ce0-477c-ac63-fb3d2725a0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:42.930169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dae44-5ce0-477c-ac63-fb3d2725a0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:42.930452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dae44-5ce0-477c-ac63-fb3d2725a0b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:42.931154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dae44-5ce0-477c-ac63-fb3d2725a0b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:42.933120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dae44-5ce0-477c-ac63-fb3d2725a0b9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:42.958895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1086/2190] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:42 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:05:42.963134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085838ea-cc5a-4e86-ad16-7ca7a3e1e667 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:42.974582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085838ea-cc5a-4e86-ad16-7ca7a3e1e667 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:42.980270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eaba1-c2a2-48c0-b133-e8f01e5a4c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:42.981211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eaba1-c2a2-48c0-b133-e8f01e5a4c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:42.981566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eaba1-c2a2-48c0-b133-e8f01e5a4c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:42.981920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eaba1-c2a2-48c0-b133-e8f01e5a4c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:42.983088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36eaba1-c2a2-48c0-b133-e8f01e5a4c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:42.985568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36eaba1-c2a2-48c0-b133-e8f01e5a4c07 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:42.997284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1105/2191] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:43.001631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dae44-5ce0-477c-ac63-fb3d2725a0b9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:43.016824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dae44-5ce0-477c-ac63-fb3d2725a0b9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:43.023896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59804f0e-130e-4736-8c1b-5bb9ab70e108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:43.024553 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59804f0e-130e-4736-8c1b-5bb9ab70e108 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.024834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59804f0e-130e-4736-8c1b-5bb9ab70e108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.025106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59804f0e-130e-4736-8c1b-5bb9ab70e108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.025834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59804f0e-130e-4736-8c1b-5bb9ab70e108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:43.027487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59804f0e-130e-4736-8c1b-5bb9ab70e108 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:43.038257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1087/2192] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:43.042486 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eaba1-c2a2-48c0-b133-e8f01e5a4c07 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:43.049158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:43.053690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eaba1-c2a2-48c0-b133-e8f01e5a4c07 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:43.060070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b5ee2-78f2-41fc-9f8f-9d939ec5a066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:43.061148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b5ee2-78f2-41fc-9f8f-9d939ec5a066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.061421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b5ee2-78f2-41fc-9f8f-9d939ec5a066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.061996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b5ee2-78f2-41fc-9f8f-9d939ec5a066 admin admin] PATH_INFO: `/v3/projects/52084318387c42afb69455b94bbd3b50/users/c577642b496549e3ba56d499b607abac/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.084127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b5ee2-78f2-41fc-9f8f-9d939ec5a066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52084318387c42afb69455b94bbd3b50, user_id=c577642b496549e3ba56d499b607abac, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:43.086890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b5ee2-78f2-41fc-9f8f-9d939ec5a066 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:43.102250 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1106/2193] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:43 2022] PUT /identity/v3/projects/52084318387c42afb69455b94bbd3b50/users/c577642b496549e3ba56d499b607abac/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:43.113284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db27ea5-dfb8-46bf-8886-7706cb50fc41 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.113714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db27ea5-dfb8-46bf-8886-7706cb50fc41 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.113824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db27ea5-dfb8-46bf-8886-7706cb50fc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.152915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0db27ea5-dfb8-46bf-8886-7706cb50fc41 None None] MFA Rules not processed for user `c577642b496549e3ba56d499b607abac`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:43.154257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0db27ea5-dfb8-46bf-8886-7706cb50fc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:43.181813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:43.225072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1088/2194] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:43.230079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b5ee2-78f2-41fc-9f8f-9d939ec5a066 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:43.296573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b5ee2-78f2-41fc-9f8f-9d939ec5a066 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:43.302800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c68c35-4dbc-4504-99dd-447739f6436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:43.303478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c68c35-4dbc-4504-99dd-447739f6436b admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.303768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c68c35-4dbc-4504-99dd-447739f6436b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.304042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c68c35-4dbc-4504-99dd-447739f6436b admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.304750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c68c35-4dbc-4504-99dd-447739f6436b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:43.306684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c68c35-4dbc-4504-99dd-447739f6436b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:43.338301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1107/2195] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:43 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:43.340140 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db27ea5-dfb8-46bf-8886-7706cb50fc41 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:43.356014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db27ea5-dfb8-46bf-8886-7706cb50fc41 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:43.415344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df2e17-b6c6-4baf-895a-61506834d1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:43.416014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df2e17-b6c6-4baf-895a-61506834d1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.416318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df2e17-b6c6-4baf-895a-61506834d1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.416601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df2e17-b6c6-4baf-895a-61506834d1fe admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.417324 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df2e17-b6c6-4baf-895a-61506834d1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:43.419401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df2e17-b6c6-4baf-895a-61506834d1fe admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:43.441080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1089/2196] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:43 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:43.445829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c68c35-4dbc-4504-99dd-447739f6436b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:43.456424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c68c35-4dbc-4504-99dd-447739f6436b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:43.461889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eed01c-32eb-4fe1-8f69-51b77813a499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:43.462586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eed01c-32eb-4fe1-8f69-51b77813a499 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.462852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eed01c-32eb-4fe1-8f69-51b77813a499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.463133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eed01c-32eb-4fe1-8f69-51b77813a499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.463866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eed01c-32eb-4fe1-8f69-51b77813a499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:43.465496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eed01c-32eb-4fe1-8f69-51b77813a499 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:43.477124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1108/2197] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:43.481337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df2e17-b6c6-4baf-895a-61506834d1fe admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:43.492129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df2e17-b6c6-4baf-895a-61506834d1fe admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:43.498142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502603ce-90f8-4463-8078-e4eda33016d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:43.499032 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502603ce-90f8-4463-8078-e4eda33016d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.499324 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502603ce-90f8-4463-8078-e4eda33016d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.499590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502603ce-90f8-4463-8078-e4eda33016d2 admin admin] PATH_INFO: `/v3/projects/234666e90dab440cb969534b73491ca5/users/eaaa0f5a0c4e4028add0e28f7207380f/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.516548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502603ce-90f8-4463-8078-e4eda33016d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=234666e90dab440cb969534b73491ca5, user_id=eaaa0f5a0c4e4028add0e28f7207380f, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:43.519250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502603ce-90f8-4463-8078-e4eda33016d2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:43.534576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1090/2198] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:43 2022] PUT /identity/v3/projects/234666e90dab440cb969534b73491ca5/users/eaaa0f5a0c4e4028add0e28f7207380f/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:43.538453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eed01c-32eb-4fe1-8f69-51b77813a499 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:43.548745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eed01c-32eb-4fe1-8f69-51b77813a499 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:43.609588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a943fc-5a86-4137-bbd8-3cbe2001529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:43.610294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a943fc-5a86-4137-bbd8-3cbe2001529f admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.610579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a943fc-5a86-4137-bbd8-3cbe2001529f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.610854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a943fc-5a86-4137-bbd8-3cbe2001529f admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.611596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a943fc-5a86-4137-bbd8-3cbe2001529f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:43.613188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a943fc-5a86-4137-bbd8-3cbe2001529f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:43.622973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1109/2199] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:43.627169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502603ce-90f8-4463-8078-e4eda33016d2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:43.637895 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502603ce-90f8-4463-8078-e4eda33016d2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:43.643304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d480f76-e9f4-42d3-b384-cab2875719bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:43.644262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d480f76-e9f4-42d3-b384-cab2875719bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.644530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d480f76-e9f4-42d3-b384-cab2875719bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.644798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d480f76-e9f4-42d3-b384-cab2875719bf admin admin] PATH_INFO: `/v3/projects/234666e90dab440cb969534b73491ca5/users/eaaa0f5a0c4e4028add0e28f7207380f/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.647156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d480f76-e9f4-42d3-b384-cab2875719bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=234666e90dab440cb969534b73491ca5, user_id=eaaa0f5a0c4e4028add0e28f7207380f, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:43.649832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d480f76-e9f4-42d3-b384-cab2875719bf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:43.660450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1091/2200] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:43 2022] PUT /identity/v3/projects/234666e90dab440cb969534b73491ca5/users/eaaa0f5a0c4e4028add0e28f7207380f/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:05:43.672385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990dec-76d7-478c-9612-22f0e99d2186 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.672819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990dec-76d7-478c-9612-22f0e99d2186 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.673234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990dec-76d7-478c-9612-22f0e99d2186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.703604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-69990dec-76d7-478c-9612-22f0e99d2186 None None] MFA Rules not processed for user `eaaa0f5a0c4e4028add0e28f7207380f`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:43.705128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-69990dec-76d7-478c-9612-22f0e99d2186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:43.751181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:43.782366 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1110/2201] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:43.797563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d480f76-e9f4-42d3-b384-cab2875719bf admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:43.800549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3d480f76-e9f4-42d3-b384-cab2875719bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:43.861443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d480f76-e9f4-42d3-b384-cab2875719bf admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:43.865874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185b91a-d6d8-430f-9751-d79a4d75f915 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:43.866601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185b91a-d6d8-430f-9751-d79a4d75f915 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.866881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185b91a-d6d8-430f-9751-d79a4d75f915 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.867269 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185b91a-d6d8-430f-9751-d79a4d75f915 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:43.876848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185b91a-d6d8-430f-9751-d79a4d75f915 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:43.878387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185b91a-d6d8-430f-9751-d79a4d75f915 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:43.888350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1092/2202] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:43.968868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990dec-76d7-478c-9612-22f0e99d2186 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:43.978801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990dec-76d7-478c-9612-22f0e99d2186 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:43.982410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadfef5-c123-43d7-8fb7-c6998079512c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:43.983065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadfef5-c123-43d7-8fb7-c6998079512c service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:43.983350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadfef5-c123-43d7-8fb7-c6998079512c service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:43.983628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadfef5-c123-43d7-8fb7-c6998079512c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:44.049971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadfef5-c123-43d7-8fb7-c6998079512c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:44.052433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadfef5-c123-43d7-8fb7-c6998079512c service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:44.064585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1111/2203] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:05:52.856886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01790427-750c-4daa-8cc5-51481d6a7e65 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:52.857419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01790427-750c-4daa-8cc5-51481d6a7e65 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:52.857419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01790427-750c-4daa-8cc5-51481d6a7e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:52.876801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-01790427-750c-4daa-8cc5-51481d6a7e65 None None] MFA Rules not processed for user `a96fa98fa00c4de29a9bed22ed9e597b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:52.878067 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-01790427-750c-4daa-8cc5-51481d6a7e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:52.896813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:52.941463 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1093/2204] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:54.059484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=39 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:54.254358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadfef5-c123-43d7-8fb7-c6998079512c service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:54.254358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2fadfef5-c123-43d7-8fb7-c6998079512c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:54.318113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadfef5-c123-43d7-8fb7-c6998079512c service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:54.323568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:54.324626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:54.324913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:54.325191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] PATH_INFO: `/v3/users/f227d722919d4d4e8afd14491d95eb55` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:54.326428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f227d722919d4d4e8afd14491d95eb55)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:54.328760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:54.376027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f227d722919d4d4e8afd14491d95eb55'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:54.383592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f227d722919d4d4e8afd14491d95eb55'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:54.384339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f227d722919d4d4e8afd14491d95eb55'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:54.384921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f227d722919d4d4e8afd14491d95eb55'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:54.397363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f227d722919d4d4e8afd14491d95eb55'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:54.411861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1112/2205] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:54 2022] DELETE /identity/v3/users/f227d722919d4d4e8afd14491d95eb55 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:54.416682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c11a1-7d73-40e5-962c-d43f72f3d98b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:54.417171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c11a1-7d73-40e5-962c-d43f72f3d98b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:54.417171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c11a1-7d73-40e5-962c-d43f72f3d98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:54.435031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-af3c11a1-7d73-40e5-962c-d43f72f3d98b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:54.436233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-af3c11a1-7d73-40e5-962c-d43f72f3d98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:54.498764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1094/2206] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:05:54.509388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:54.511129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:54.570429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b581e-1f30-45ba-a5d4-cf5a23d14523 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:54.574059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be410268-ff0b-4d7d-8554-11b7aa34a04c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:54.574751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be410268-ff0b-4d7d-8554-11b7aa34a04c service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:54.575035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be410268-ff0b-4d7d-8554-11b7aa34a04c service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:54.575314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be410268-ff0b-4d7d-8554-11b7aa34a04c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:54.585286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be410268-ff0b-4d7d-8554-11b7aa34a04c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:54.586893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be410268-ff0b-4d7d-8554-11b7aa34a04c service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:54.599207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1113/2207] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:05:54.902308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c11a1-7d73-40e5-962c-d43f72f3d98b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:54.903844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-af3c11a1-7d73-40e5-962c-d43f72f3d98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:54.917458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c11a1-7d73-40e5-962c-d43f72f3d98b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:54.923530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:54.924441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:54.924743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:54.925023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] PATH_INFO: `/v3/projects/475fec527f064cfabe7733a58d9d9a34` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:54.926391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=475fec527f064cfabe7733a58d9d9a34)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:54.928355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:54.978112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '475fec527f064cfabe7733a58d9d9a34', 'request_id': 'req-403e5800-e591-442d-8d0b-d8e99e75bfc0'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:54.985757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '475fec527f064cfabe7733a58d9d9a34', 'request_id': 'req-403e5800-e591-442d-8d0b-d8e99e75bfc0'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:54.991622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '475fec527f064cfabe7733a58d9d9a34', 'request_id': 'req-403e5800-e591-442d-8d0b-d8e99e75bfc0'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.003899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] The token cache is being invalidate because project 475fec527f064cfabe7733a58d9d9a34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:55.004190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.004947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.006680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1095/2208] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:54 2022] DELETE /identity/v3/projects/475fec527f064cfabe7733a58d9d9a34 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:55.010904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be410268-ff0b-4d7d-8554-11b7aa34a04c service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:55.020621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be410268-ff0b-4d7d-8554-11b7aa34a04c service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:55.038464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:55.080829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:55.081720 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:55.082025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:55.082321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] PATH_INFO: `/v3/users/8b2cc7e700374279af9a372b83358124` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:55.083523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2cc7e700374279af9a372b83358124)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:55.085453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:55.134264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2cc7e700374279af9a372b83358124'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.142722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2cc7e700374279af9a372b83358124'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.143329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2cc7e700374279af9a372b83358124'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.143943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2cc7e700374279af9a372b83358124'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.156357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2cc7e700374279af9a372b83358124'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.168666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1114/2209] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:55 2022] DELETE /identity/v3/users/8b2cc7e700374279af9a372b83358124 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:55.450390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:55.451932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:55.473687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5800-e591-442d-8d0b-d8e99e75bfc0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:55.525272 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:55.536739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:55.537628 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:55.538073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:55.538720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] PATH_INFO: `/v3/projects/0fd04ca3ab72418abfec2648373140cc` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:55.540152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd04ca3ab72418abfec2648373140cc)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:55.541983 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:55.591616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd04ca3ab72418abfec2648373140cc', 'request_id': 'req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.600144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd04ca3ab72418abfec2648373140cc', 'request_id': 'req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.606171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd04ca3ab72418abfec2648373140cc', 'request_id': 'req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.617702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] The token cache is being invalidate because project 0fd04ca3ab72418abfec2648373140cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:55.617977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.618802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.620447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1096/2210] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:55 2022] DELETE /identity/v3/projects/0fd04ca3ab72418abfec2648373140cc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:55.624330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:55.625874 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:55.639561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a330859-ac3d-43f5-8678-37af660e068a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:55.703581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:55.704512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:55.704783 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:55.705435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] PATH_INFO: `/v3/users/cc3e2a9928674c84bd4ff5daa2dcf7f3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:55.707356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc3e2a9928674c84bd4ff5daa2dcf7f3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:55.709094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:55.765416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:55.769464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc3e2a9928674c84bd4ff5daa2dcf7f3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.790466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc3e2a9928674c84bd4ff5daa2dcf7f3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.791659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc3e2a9928674c84bd4ff5daa2dcf7f3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.792353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc3e2a9928674c84bd4ff5daa2dcf7f3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.804397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-609fca78-a293-4106-bd85-701abe0c1208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc3e2a9928674c84bd4ff5daa2dcf7f3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:55.816632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1115/2211] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:05:55 2022] DELETE /identity/v3/users/cc3e2a9928674c84bd4ff5daa2dcf7f3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:56.093400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:56.094868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:56.108216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3d93e-003b-4ec8-aab4-ee87c5bcb35d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:56.166400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:56.167311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.167594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.167868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] PATH_INFO: `/v3/projects/1ede1ea9d8f146399497d7028cc62962` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.169198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ede1ea9d8f146399497d7028cc62962)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:56.171178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:56.222508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ede1ea9d8f146399497d7028cc62962', 'request_id': 'req-37eb34a7-33b7-45dc-832b-1de95142d59d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:56.231705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ede1ea9d8f146399497d7028cc62962', 'request_id': 'req-37eb34a7-33b7-45dc-832b-1de95142d59d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:56.238117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ede1ea9d8f146399497d7028cc62962', 'request_id': 'req-37eb34a7-33b7-45dc-832b-1de95142d59d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:56.250276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] The token cache is being invalidate because project 1ede1ea9d8f146399497d7028cc62962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:05:56.250560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:56.251425 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-37eb34a7-33b7-45dc-832b-1de95142d59d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:05:56.253055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1097/2212] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:05:56 2022] DELETE /identity/v3/projects/1ede1ea9d8f146399497d7028cc62962 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:56.258269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf0f4-6f89-43bf-827f-c00937808702 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.259065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf0f4-6f89-43bf-827f-c00937808702 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.259373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf0f4-6f89-43bf-827f-c00937808702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.278046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3abbf0f4-6f89-43bf-827f-c00937808702 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:56.279283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3abbf0f4-6f89-43bf-827f-c00937808702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:56.341697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1116/2213] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:56.349358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dc941-0c8e-43ed-9bfe-cfeb0e6ea837 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.349646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dc941-0c8e-43ed-9bfe-cfeb0e6ea837 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.349924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dc941-0c8e-43ed-9bfe-cfeb0e6ea837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.369216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d82dc941-0c8e-43ed-9bfe-cfeb0e6ea837 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:56.370424 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d82dc941-0c8e-43ed-9bfe-cfeb0e6ea837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:56.378888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1098/2214] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:56.383064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbf0f4-6f89-43bf-827f-c00937808702 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:56.393448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbf0f4-6f89-43bf-827f-c00937808702 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:56.399455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d55477-ae07-4ff7-8ad7-97818bd286cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:56.400229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d55477-ae07-4ff7-8ad7-97818bd286cd admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.400577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d55477-ae07-4ff7-8ad7-97818bd286cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.400859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d55477-ae07-4ff7-8ad7-97818bd286cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.401594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d55477-ae07-4ff7-8ad7-97818bd286cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:05:56.401910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d55477-ae07-4ff7-8ad7-97818bd286cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:56.403589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d55477-ae07-4ff7-8ad7-97818bd286cd admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:56.415593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1117/2215] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:05:56.419872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dc941-0c8e-43ed-9bfe-cfeb0e6ea837 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:56.425626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:56.429405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dc941-0c8e-43ed-9bfe-cfeb0e6ea837 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:56.435538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab81894-0eba-4cc7-92ee-e083000c4219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:56.436524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab81894-0eba-4cc7-92ee-e083000c4219 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.436862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab81894-0eba-4cc7-92ee-e083000c4219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.437197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab81894-0eba-4cc7-92ee-e083000c4219 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.437942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab81894-0eba-4cc7-92ee-e083000c4219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:56.440535 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab81894-0eba-4cc7-92ee-e083000c4219 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:56.470075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1099/2216] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:05:56 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:56.473751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d55477-ae07-4ff7-8ad7-97818bd286cd admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:56.483797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d55477-ae07-4ff7-8ad7-97818bd286cd admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:56.550318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dbe53-430d-4f12-807c-355685f47f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:56.551146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dbe53-430d-4f12-807c-355685f47f28 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.551312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dbe53-430d-4f12-807c-355685f47f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.551551 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21dbe53-430d-4f12-807c-355685f47f28 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.552275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21dbe53-430d-4f12-807c-355685f47f28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:56.554256 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21dbe53-430d-4f12-807c-355685f47f28 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:56.576190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1118/2217] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:05:56 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:05:56.580446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab81894-0eba-4cc7-92ee-e083000c4219 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:56.591156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab81894-0eba-4cc7-92ee-e083000c4219 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:56.596257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c974f5-061a-445c-a123-ad75c951d1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:56.596931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c974f5-061a-445c-a123-ad75c951d1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.597211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c974f5-061a-445c-a123-ad75c951d1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.597488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c974f5-061a-445c-a123-ad75c951d1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.598286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c974f5-061a-445c-a123-ad75c951d1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:56.599907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c974f5-061a-445c-a123-ad75c951d1c0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:56.611614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1100/2218] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:56.615695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dbe53-430d-4f12-807c-355685f47f28 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:56.627025 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21dbe53-430d-4f12-807c-355685f47f28 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:56.632080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a594572-1f97-4df9-92a6-29b718a400e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:56.632737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a594572-1f97-4df9-92a6-29b718a400e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.633018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a594572-1f97-4df9-92a6-29b718a400e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.633373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a594572-1f97-4df9-92a6-29b718a400e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.634190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a594572-1f97-4df9-92a6-29b718a400e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:56.635862 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a594572-1f97-4df9-92a6-29b718a400e1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:56.646728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1119/2219] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:05:56.652087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c974f5-061a-445c-a123-ad75c951d1c0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:56.665135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c974f5-061a-445c-a123-ad75c951d1c0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:56.671005 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d63d6e-0956-40d8-afad-7498cf8e626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:56.671892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d63d6e-0956-40d8-afad-7498cf8e626b admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.672173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d63d6e-0956-40d8-afad-7498cf8e626b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.672448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d63d6e-0956-40d8-afad-7498cf8e626b admin admin] PATH_INFO: `/v3/projects/73b120df0997427bbdaedb45b7da1968/users/1d254e641c3f473b99b89ca4554b7f3a/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.688001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d63d6e-0956-40d8-afad-7498cf8e626b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b120df0997427bbdaedb45b7da1968, user_id=1d254e641c3f473b99b89ca4554b7f3a, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:56.690732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d63d6e-0956-40d8-afad-7498cf8e626b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:56.705389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1101/2220] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:05:56 2022] PUT /identity/v3/projects/73b120df0997427bbdaedb45b7da1968/users/1d254e641c3f473b99b89ca4554b7f3a/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:05:56.716429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4f8dc-2873-4214-a556-1aa9c02e21d3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.716724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4f8dc-2873-4214-a556-1aa9c02e21d3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.717028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4f8dc-2873-4214-a556-1aa9c02e21d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.745792 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bbd4f8dc-2873-4214-a556-1aa9c02e21d3 None None] MFA Rules not processed for user `1d254e641c3f473b99b89ca4554b7f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:05:56.747773 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bbd4f8dc-2873-4214-a556-1aa9c02e21d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:56.769033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:05:56.831587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1120/2221] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:05:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:05:56.843411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d63d6e-0956-40d8-afad-7498cf8e626b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:05:56.845103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-46d63d6e-0956-40d8-afad-7498cf8e626b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:05:56.913029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d63d6e-0956-40d8-afad-7498cf8e626b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:05:56.918360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a3cc0-2b39-4fe2-bc82-7ced60874d89 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bf4d9571d541bcbba87094198dc2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:05:56.919037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a3cc0-2b39-4fe2-bc82-7ced60874d89 service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:05:56.919318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a3cc0-2b39-4fe2-bc82-7ced60874d89 service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:05:56.919597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a3cc0-2b39-4fe2-bc82-7ced60874d89 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:05:56.931351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a3cc0-2b39-4fe2-bc82-7ced60874d89 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:05:56.932824 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a3cc0-2b39-4fe2-bc82-7ced60874d89 service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:05:56.942673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1102/2222] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:05:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:06:00.653173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4f8dc-2873-4214-a556-1aa9c02e21d3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:00.654680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bbd4f8dc-2873-4214-a556-1aa9c02e21d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:00.731478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4f8dc-2873-4214-a556-1aa9c02e21d3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:00.735967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:00.736851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:00.737177 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:00.737454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] PATH_INFO: `/v3/users/2a4f8542e17945ae993984e2b03881c5` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:00.738714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4f8542e17945ae993984e2b03881c5)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:00.740651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:00.796198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4f8542e17945ae993984e2b03881c5'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:00.805538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4f8542e17945ae993984e2b03881c5'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:00.806378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4f8542e17945ae993984e2b03881c5'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:00.807368 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4f8542e17945ae993984e2b03881c5'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:00.822804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4f8542e17945ae993984e2b03881c5'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:00.835966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1121/2223] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:06:00 2022] DELETE /identity/v3/users/2a4f8542e17945ae993984e2b03881c5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:00.840964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0f267-4f7c-4a10-b0ea-427e4e3e6688 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:00.841846 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0f267-4f7c-4a10-b0ea-427e4e3e6688 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:00.842343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0f267-4f7c-4a10-b0ea-427e4e3e6688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:00.861042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-feb0f267-4f7c-4a10-b0ea-427e4e3e6688 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:00.862388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-feb0f267-4f7c-4a10-b0ea-427e4e3e6688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:00.932223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1103/2224] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:00.942103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:00.943594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:00.985494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:01.011017 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f77f2-5987-4015-bc94-8a3ec1275f1c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:01.014932 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2a4db-3202-4057-8239-09b45790078f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:01.015574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2a4db-3202-4057-8239-09b45790078f service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.015892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2a4db-3202-4057-8239-09b45790078f service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.016170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2a4db-3202-4057-8239-09b45790078f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:01.027332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2a4db-3202-4057-8239-09b45790078f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:01.028993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2a4db-3202-4057-8239-09b45790078f service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:01.039990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1122/2225] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:06:01.313504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0f267-4f7c-4a10-b0ea-427e4e3e6688 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:01.315014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-feb0f267-4f7c-4a10-b0ea-427e4e3e6688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:01.329951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0f267-4f7c-4a10-b0ea-427e4e3e6688 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:01.335744 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:01.336650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.336916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.337201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] PATH_INFO: `/v3/projects/e0a09ec970db4f6182e567ac40cc6e57` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:01.338907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0a09ec970db4f6182e567ac40cc6e57)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:01.340761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:01.388174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:01.393349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0a09ec970db4f6182e567ac40cc6e57', 'request_id': 'req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:01.404282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0a09ec970db4f6182e567ac40cc6e57', 'request_id': 'req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:01.411093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0a09ec970db4f6182e567ac40cc6e57', 'request_id': 'req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:01.424706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] The token cache is being invalidate because project e0a09ec970db4f6182e567ac40cc6e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:06:01.424978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:01.425781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-9ff2afc5-d1f8-4a22-ad94-0081d41deecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:01.427535 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1104/2226] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:06:01 2022] DELETE /identity/v3/projects/e0a09ec970db4f6182e567ac40cc6e57 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:01.433119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645c638-310e-4b34-ac5d-45646b83f95b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.433589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645c638-310e-4b34-ac5d-45646b83f95b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.433695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645c638-310e-4b34-ac5d-45646b83f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:01.453909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3645c638-310e-4b34-ac5d-45646b83f95b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:01.455108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3645c638-310e-4b34-ac5d-45646b83f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:01.528607 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1123/2227] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:01.537358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d713f5-6f3b-46f4-8d95-4f2c7a98fc22 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.537649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d713f5-6f3b-46f4-8d95-4f2c7a98fc22 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.537948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d713f5-6f3b-46f4-8d95-4f2c7a98fc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:01.562404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e8d713f5-6f3b-46f4-8d95-4f2c7a98fc22 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:01.563626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e8d713f5-6f3b-46f4-8d95-4f2c7a98fc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:01.573707 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1105/2228] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:01.578291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645c638-310e-4b34-ac5d-45646b83f95b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:01.589323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645c638-310e-4b34-ac5d-45646b83f95b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:01.595659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495da653-be43-4b6e-a50c-4d3b501fd9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:01.596446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495da653-be43-4b6e-a50c-4d3b501fd9ea admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.596791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495da653-be43-4b6e-a50c-4d3b501fd9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.597141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495da653-be43-4b6e-a50c-4d3b501fd9ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:01.598190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495da653-be43-4b6e-a50c-4d3b501fd9ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:06:01.598506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495da653-be43-4b6e-a50c-4d3b501fd9ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:01.600200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495da653-be43-4b6e-a50c-4d3b501fd9ea admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:01.624383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1124/2229] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:06:01.629037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d713f5-6f3b-46f4-8d95-4f2c7a98fc22 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:01.639620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d713f5-6f3b-46f4-8d95-4f2c7a98fc22 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:01.644703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ffc35-85a6-4b4a-8d59-38ff7965fe44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:01.645428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ffc35-85a6-4b4a-8d59-38ff7965fe44 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.645871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ffc35-85a6-4b4a-8d59-38ff7965fe44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.646176 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ffc35-85a6-4b4a-8d59-38ff7965fe44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:01.647263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ffc35-85a6-4b4a-8d59-38ff7965fe44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:01.649811 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ffc35-85a6-4b4a-8d59-38ff7965fe44 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:01.681232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1106/2230] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:06:01 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:01.682498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495da653-be43-4b6e-a50c-4d3b501fd9ea admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:01.692542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495da653-be43-4b6e-a50c-4d3b501fd9ea admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:01.763376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df8b63-c4aa-4671-9f64-80710bf64685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:01.764076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df8b63-c4aa-4671-9f64-80710bf64685 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.764361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df8b63-c4aa-4671-9f64-80710bf64685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.764636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df8b63-c4aa-4671-9f64-80710bf64685 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:01.765342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df8b63-c4aa-4671-9f64-80710bf64685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:01.767538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df8b63-c4aa-4671-9f64-80710bf64685 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:01.801801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1125/2231] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:06:01 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:01.802982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ffc35-85a6-4b4a-8d59-38ff7965fe44 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:01.816238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ffc35-85a6-4b4a-8d59-38ff7965fe44 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:01.834294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fff3e-062e-4e53-9a7f-912d30208a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:01.834771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fff3e-062e-4e53-9a7f-912d30208a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.835052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fff3e-062e-4e53-9a7f-912d30208a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.835331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fff3e-062e-4e53-9a7f-912d30208a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:01.836107 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fff3e-062e-4e53-9a7f-912d30208a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:01.837819 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fff3e-062e-4e53-9a7f-912d30208a00 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:01.850564 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1107/2232] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:01.854155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df8b63-c4aa-4671-9f64-80710bf64685 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:01.865221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df8b63-c4aa-4671-9f64-80710bf64685 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:01.870038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f128b40-e80a-4260-94dc-4f59b24cdf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:01.870741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f128b40-e80a-4260-94dc-4f59b24cdf68 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.871030 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f128b40-e80a-4260-94dc-4f59b24cdf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.871314 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f128b40-e80a-4260-94dc-4f59b24cdf68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:01.872048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f128b40-e80a-4260-94dc-4f59b24cdf68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:01.873677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f128b40-e80a-4260-94dc-4f59b24cdf68 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:01.890139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1126/2233] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:01.891537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fff3e-062e-4e53-9a7f-912d30208a00 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:01.907349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fff3e-062e-4e53-9a7f-912d30208a00 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:01.914849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba6468-d330-4057-90c9-78f059108f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:01.918621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba6468-d330-4057-90c9-78f059108f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.919127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba6468-d330-4057-90c9-78f059108f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.919854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba6468-d330-4057-90c9-78f059108f24 admin admin] PATH_INFO: `/v3/projects/c5a6cbaf7bec48bea8f3d2f0fabed14f/users/106a93182fbd4a1682ddcbe2cc2e5f3d/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:01.937766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba6468-d330-4057-90c9-78f059108f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a6cbaf7bec48bea8f3d2f0fabed14f, user_id=106a93182fbd4a1682ddcbe2cc2e5f3d, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:01.940679 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba6468-d330-4057-90c9-78f059108f24 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:01.956690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1108/2234] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:06:01 2022] PUT /identity/v3/projects/c5a6cbaf7bec48bea8f3d2f0fabed14f/users/106a93182fbd4a1682ddcbe2cc2e5f3d/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:01.968182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7d8fd-affd-4bbf-8869-391bf88729dd None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:01.968482 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7d8fd-affd-4bbf-8869-391bf88729dd None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:01.968762 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7d8fd-affd-4bbf-8869-391bf88729dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:02.002824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-32f7d8fd-affd-4bbf-8869-391bf88729dd None None] MFA Rules not processed for user `106a93182fbd4a1682ddcbe2cc2e5f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:02.004271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-32f7d8fd-affd-4bbf-8869-391bf88729dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:02.038077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:02.077912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1127/2235] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:02.090631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba6468-d330-4057-90c9-78f059108f24 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:02.092401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d7ba6468-d330-4057-90c9-78f059108f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:02.167879 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba6468-d330-4057-90c9-78f059108f24 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:02.171710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f55cc-7a9c-4ef0-b0d5-79294f8c9a7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:02.172698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f55cc-7a9c-4ef0-b0d5-79294f8c9a7c service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:02.173233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f55cc-7a9c-4ef0-b0d5-79294f8c9a7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:02.173714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f55cc-7a9c-4ef0-b0d5-79294f8c9a7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:02.184729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f55cc-7a9c-4ef0-b0d5-79294f8c9a7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:02.187146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f55cc-7a9c-4ef0-b0d5-79294f8c9a7c service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:02.201682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1109/2236] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:06:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:06:12.074838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=12 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:12.195180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:12.299763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7d8fd-affd-4bbf-8869-391bf88729dd None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:12.370331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7d8fd-affd-4bbf-8869-391bf88729dd None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:12.376666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:12.377561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:12.377848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:12.378126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] PATH_INFO: `/v3/users/106a93182fbd4a1682ddcbe2cc2e5f3d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:12.379540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=106a93182fbd4a1682ddcbe2cc2e5f3d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:12.381537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:12.434682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '106a93182fbd4a1682ddcbe2cc2e5f3d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:12.443713 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '106a93182fbd4a1682ddcbe2cc2e5f3d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:12.444710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '106a93182fbd4a1682ddcbe2cc2e5f3d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:12.445381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '106a93182fbd4a1682ddcbe2cc2e5f3d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:12.458026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '106a93182fbd4a1682ddcbe2cc2e5f3d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:12.478153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1128/2237] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:06:12 2022] DELETE /identity/v3/users/106a93182fbd4a1682ddcbe2cc2e5f3d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:12.480084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544f21f-ecca-4dcc-b34b-0ad46028235f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:12.480337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544f21f-ecca-4dcc-b34b-0ad46028235f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:12.480616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544f21f-ecca-4dcc-b34b-0ad46028235f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:12.499623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9544f21f-ecca-4dcc-b34b-0ad46028235f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:12.500862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9544f21f-ecca-4dcc-b34b-0ad46028235f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:12.565808 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1110/2238] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:12.576077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:12.577974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:12.640747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95755a-5c77-4780-a94d-63ffe40319d3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:12.645083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1888d6f-f2d0-47c3-8314-3ab83d75190f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:12.645745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1888d6f-f2d0-47c3-8314-3ab83d75190f service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:12.646031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1888d6f-f2d0-47c3-8314-3ab83d75190f service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:12.646342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1888d6f-f2d0-47c3-8314-3ab83d75190f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:12.655873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1888d6f-f2d0-47c3-8314-3ab83d75190f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:12.657323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1888d6f-f2d0-47c3-8314-3ab83d75190f service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:12.667553 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1129/2239] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:06:12.978778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544f21f-ecca-4dcc-b34b-0ad46028235f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:12.980382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9544f21f-ecca-4dcc-b34b-0ad46028235f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:13.006945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544f21f-ecca-4dcc-b34b-0ad46028235f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:13.018133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:13.019117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.019433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.019713 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] PATH_INFO: `/v3/projects/c5a6cbaf7bec48bea8f3d2f0fabed14f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.020977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5a6cbaf7bec48bea8f3d2f0fabed14f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:13.022970 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:13.077720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5a6cbaf7bec48bea8f3d2f0fabed14f', 'request_id': 'req-b382319b-0ede-45b7-bd08-0a0f25620944'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:13.089132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5a6cbaf7bec48bea8f3d2f0fabed14f', 'request_id': 'req-b382319b-0ede-45b7-bd08-0a0f25620944'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:13.095847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5a6cbaf7bec48bea8f3d2f0fabed14f', 'request_id': 'req-b382319b-0ede-45b7-bd08-0a0f25620944'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:13.107255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] The token cache is being invalidate because project c5a6cbaf7bec48bea8f3d2f0fabed14f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:06:13.107539 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:13.108356 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b382319b-0ede-45b7-bd08-0a0f25620944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:13.110336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1111/2240] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:06:12 2022] DELETE /identity/v3/projects/c5a6cbaf7bec48bea8f3d2f0fabed14f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:13.115417 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b45f3b-0c66-4636-9199-ff7a57ce526f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.115854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b45f3b-0c66-4636-9199-ff7a57ce526f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.116094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b45f3b-0c66-4636-9199-ff7a57ce526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.135868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-47b45f3b-0c66-4636-9199-ff7a57ce526f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:13.137107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-47b45f3b-0c66-4636-9199-ff7a57ce526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:13.159605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:13.216134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1130/2241] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:06:13.228585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f68be7-0ac4-41f1-bb32-d411fe1ae90c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.229101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f68be7-0ac4-41f1-bb32-d411fe1ae90c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.229192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f68be7-0ac4-41f1-bb32-d411fe1ae90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.249260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-72f68be7-0ac4-41f1-bb32-d411fe1ae90c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:13.250887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-72f68be7-0ac4-41f1-bb32-d411fe1ae90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:13.260527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1112/2242] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:13.264897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b45f3b-0c66-4636-9199-ff7a57ce526f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:13.276628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b45f3b-0c66-4636-9199-ff7a57ce526f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:13.283240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16965b9a-54a4-447b-bc87-f5ee38fedbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:13.283932 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16965b9a-54a4-447b-bc87-f5ee38fedbdf admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.284217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16965b9a-54a4-447b-bc87-f5ee38fedbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.284495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16965b9a-54a4-447b-bc87-f5ee38fedbdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.285237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16965b9a-54a4-447b-bc87-f5ee38fedbdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:06:13.285608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16965b9a-54a4-447b-bc87-f5ee38fedbdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:13.287407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16965b9a-54a4-447b-bc87-f5ee38fedbdf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:13.310169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1131/2243] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:06:13.312821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f68be7-0ac4-41f1-bb32-d411fe1ae90c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:13.324316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f68be7-0ac4-41f1-bb32-d411fe1ae90c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:13.333997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cdd88-3360-436a-b6ef-8e4cb956cd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:13.334768 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cdd88-3360-436a-b6ef-8e4cb956cd9c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.335065 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cdd88-3360-436a-b6ef-8e4cb956cd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.335333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cdd88-3360-436a-b6ef-8e4cb956cd9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.336050 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cdd88-3360-436a-b6ef-8e4cb956cd9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:13.337993 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cdd88-3360-436a-b6ef-8e4cb956cd9c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:13.364588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1113/2244] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:06:13 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:13.368877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16965b9a-54a4-447b-bc87-f5ee38fedbdf admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:13.379435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16965b9a-54a4-447b-bc87-f5ee38fedbdf admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:13.437148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476b009-0d72-45b3-80d5-af93ee3d2c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:13.437752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476b009-0d72-45b3-80d5-af93ee3d2c70 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.438020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476b009-0d72-45b3-80d5-af93ee3d2c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.438328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476b009-0d72-45b3-80d5-af93ee3d2c70 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.439058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476b009-0d72-45b3-80d5-af93ee3d2c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:13.440995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476b009-0d72-45b3-80d5-af93ee3d2c70 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:13.463668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1132/2245] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:06:13 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:06:13.468443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cdd88-3360-436a-b6ef-8e4cb956cd9c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:13.479105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cdd88-3360-436a-b6ef-8e4cb956cd9c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:13.484046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33fd92-dae1-4444-b120-e27246bd4512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:13.484720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33fd92-dae1-4444-b120-e27246bd4512 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.485002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33fd92-dae1-4444-b120-e27246bd4512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.485280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33fd92-dae1-4444-b120-e27246bd4512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.486038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33fd92-dae1-4444-b120-e27246bd4512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:13.487686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33fd92-dae1-4444-b120-e27246bd4512 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:13.500603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1114/2246] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:13.505058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476b009-0d72-45b3-80d5-af93ee3d2c70 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:13.515795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476b009-0d72-45b3-80d5-af93ee3d2c70 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:13.521536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313fc53-4f68-4cbc-821d-410040d31bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:13.522293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313fc53-4f68-4cbc-821d-410040d31bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.522578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313fc53-4f68-4cbc-821d-410040d31bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.522836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313fc53-4f68-4cbc-821d-410040d31bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.523572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313fc53-4f68-4cbc-821d-410040d31bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:13.525237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313fc53-4f68-4cbc-821d-410040d31bc8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:13.536036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1133/2247] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:06:13.540267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33fd92-dae1-4444-b120-e27246bd4512 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:13.550794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33fd92-dae1-4444-b120-e27246bd4512 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:13.556347 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24157d-9f6a-4fe2-93f1-c806647bae99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:13.557259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24157d-9f6a-4fe2-93f1-c806647bae99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.557514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24157d-9f6a-4fe2-93f1-c806647bae99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.557805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24157d-9f6a-4fe2-93f1-c806647bae99 admin admin] PATH_INFO: `/v3/projects/17bb5c3721fc4505a51879dae4762b43/users/ee21ef9eacd24177a852f28d17376191/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.575299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24157d-9f6a-4fe2-93f1-c806647bae99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bb5c3721fc4505a51879dae4762b43, user_id=ee21ef9eacd24177a852f28d17376191, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:13.577968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24157d-9f6a-4fe2-93f1-c806647bae99 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:13.593880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1115/2248] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:06:13 2022] PUT /identity/v3/projects/17bb5c3721fc4505a51879dae4762b43/users/ee21ef9eacd24177a852f28d17376191/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:13.599227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f3b6-0f74-44e8-9cfe-07a8bcb7049e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.599521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f3b6-0f74-44e8-9cfe-07a8bcb7049e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.599807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f3b6-0f74-44e8-9cfe-07a8bcb7049e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.631350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7414f3b6-0f74-44e8-9cfe-07a8bcb7049e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:13.632595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7414f3b6-0f74-44e8-9cfe-07a8bcb7049e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:13.700473 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1134/2249] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:06:13.711488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24157d-9f6a-4fe2-93f1-c806647bae99 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:13.713255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4a24157d-9f6a-4fe2-93f1-c806647bae99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:13.737346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:13.779902 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24157d-9f6a-4fe2-93f1-c806647bae99 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:13.784097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc864f-2be2-4161-8eab-ebe5f06aabe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:13.784911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc864f-2be2-4161-8eab-ebe5f06aabe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:13.785291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc864f-2be2-4161-8eab-ebe5f06aabe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:13.785690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc864f-2be2-4161-8eab-ebe5f06aabe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:13.801486 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc864f-2be2-4161-8eab-ebe5f06aabe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:13.803798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc864f-2be2-4161-8eab-ebe5f06aabe8 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:13.816429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1116/2250] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:06:18.332439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad65ec-a187-4b4c-84b9-5d70d8e82476 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:18.332945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad65ec-a187-4b4c-84b9-5d70d8e82476 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:18.333026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad65ec-a187-4b4c-84b9-5d70d8e82476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:18.360544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5dad65ec-a187-4b4c-84b9-5d70d8e82476 None None] MFA Rules not processed for user `ee21ef9eacd24177a852f28d17376191`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:18.361768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5dad65ec-a187-4b4c-84b9-5d70d8e82476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:18.379039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:18.448179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1135/2251] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:18.452866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc864f-2be2-4161-8eab-ebe5f06aabe8 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:18.462726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc864f-2be2-4161-8eab-ebe5f06aabe8 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:18.468251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37237e95-f49c-4781-946c-30671dce75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:18.468990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37237e95-f49c-4781-946c-30671dce75e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:18.469526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37237e95-f49c-4781-946c-30671dce75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:18.469871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37237e95-f49c-4781-946c-30671dce75e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:18.470518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37237e95-f49c-4781-946c-30671dce75e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:18.473905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37237e95-f49c-4781-946c-30671dce75e7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:18.520926 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1117/2252] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:06:18 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:18.524815 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad65ec-a187-4b4c-84b9-5d70d8e82476 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:18.536657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad65ec-a187-4b4c-84b9-5d70d8e82476 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:18.611789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c822c5d-51a6-4294-af06-76d676e4c763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:18.612499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822c5d-51a6-4294-af06-76d676e4c763 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:18.612784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822c5d-51a6-4294-af06-76d676e4c763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:18.613062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c822c5d-51a6-4294-af06-76d676e4c763 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:18.613801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c822c5d-51a6-4294-af06-76d676e4c763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:18.615844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c822c5d-51a6-4294-af06-76d676e4c763 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:18.641174 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1136/2253] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:06:18 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:18.645030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37237e95-f49c-4781-946c-30671dce75e7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:18.655892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37237e95-f49c-4781-946c-30671dce75e7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:18.662566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0d684-9b6c-475c-bcc3-74dbc0716848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:18.663330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0d684-9b6c-475c-bcc3-74dbc0716848 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:18.663617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0d684-9b6c-475c-bcc3-74dbc0716848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:18.663920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0d684-9b6c-475c-bcc3-74dbc0716848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:18.664665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0d684-9b6c-475c-bcc3-74dbc0716848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:18.666687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0d684-9b6c-475c-bcc3-74dbc0716848 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:18.678833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1118/2254] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:18.683222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c822c5d-51a6-4294-af06-76d676e4c763 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:18.694327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c822c5d-51a6-4294-af06-76d676e4c763 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:18.700376 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f17fa-5e1c-4626-b64a-0c090f3af3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:18.701297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f17fa-5e1c-4626-b64a-0c090f3af3c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:18.701576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f17fa-5e1c-4626-b64a-0c090f3af3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:18.702876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f17fa-5e1c-4626-b64a-0c090f3af3c6 admin admin] PATH_INFO: `/v3/projects/8d5fcf3b767a463083bb51756473c816/users/3217e3b38bb648cb98dafba657b8f72f/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:18.723094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f17fa-5e1c-4626-b64a-0c090f3af3c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5fcf3b767a463083bb51756473c816, user_id=3217e3b38bb648cb98dafba657b8f72f, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:18.725788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f17fa-5e1c-4626-b64a-0c090f3af3c6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:18.740295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1137/2255] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:06:18 2022] PUT /identity/v3/projects/8d5fcf3b767a463083bb51756473c816/users/3217e3b38bb648cb98dafba657b8f72f/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:18.744363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0d684-9b6c-475c-bcc3-74dbc0716848 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:18.755635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0d684-9b6c-475c-bcc3-74dbc0716848 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:18.814776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a384669-1adc-4b17-b024-326bb8403613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:18.815464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a384669-1adc-4b17-b024-326bb8403613 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:18.815747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a384669-1adc-4b17-b024-326bb8403613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:18.816025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a384669-1adc-4b17-b024-326bb8403613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:18.816766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a384669-1adc-4b17-b024-326bb8403613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:18.818454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a384669-1adc-4b17-b024-326bb8403613 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:18.829946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1119/2256] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:18.834398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f17fa-5e1c-4626-b64a-0c090f3af3c6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:18.845230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f17fa-5e1c-4626-b64a-0c090f3af3c6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:18.850049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de7ec0-c036-41d8-b3b2-2ebba46732c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:18.850989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de7ec0-c036-41d8-b3b2-2ebba46732c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:18.851271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de7ec0-c036-41d8-b3b2-2ebba46732c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:18.851549 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de7ec0-c036-41d8-b3b2-2ebba46732c4 admin admin] PATH_INFO: `/v3/projects/8d5fcf3b767a463083bb51756473c816/users/3217e3b38bb648cb98dafba657b8f72f/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:18.853562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de7ec0-c036-41d8-b3b2-2ebba46732c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5fcf3b767a463083bb51756473c816, user_id=3217e3b38bb648cb98dafba657b8f72f, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:18.856265 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de7ec0-c036-41d8-b3b2-2ebba46732c4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:18.868471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1138/2257] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:06:18 2022] PUT /identity/v3/projects/8d5fcf3b767a463083bb51756473c816/users/3217e3b38bb648cb98dafba657b8f72f/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:24.002492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20da6e-90b7-495d-a990-297cd45badc0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:24.003016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20da6e-90b7-495d-a990-297cd45badc0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:24.003108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20da6e-90b7-495d-a990-297cd45badc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:24.030706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5c20da6e-90b7-495d-a990-297cd45badc0 None None] MFA Rules not processed for user `3217e3b38bb648cb98dafba657b8f72f`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:24.031945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5c20da6e-90b7-495d-a990-297cd45badc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:24.110062 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1120/2258] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:24.124550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de7ec0-c036-41d8-b3b2-2ebba46732c4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:24.126387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d1de7ec0-c036-41d8-b3b2-2ebba46732c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:24.206844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de7ec0-c036-41d8-b3b2-2ebba46732c4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:24.212455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a62673-4c21-44ba-b169-16eeb34e23ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:24.213128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a62673-4c21-44ba-b169-16eeb34e23ba service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:24.213415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a62673-4c21-44ba-b169-16eeb34e23ba service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:24.213692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a62673-4c21-44ba-b169-16eeb34e23ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:24.220714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:24.277890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a62673-4c21-44ba-b169-16eeb34e23ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:24.279641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a62673-4c21-44ba-b169-16eeb34e23ba service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:24.289312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1139/2259] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:06:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:06:34.106693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:34.285361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=24 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:50.219613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20da6e-90b7-495d-a990-297cd45badc0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:50.221246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5c20da6e-90b7-495d-a990-297cd45badc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:50.289696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20da6e-90b7-495d-a990-297cd45badc0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:50.296069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:50.296966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:50.297247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:50.297513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] PATH_INFO: `/v3/users/c577642b496549e3ba56d499b607abac` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:50.298734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c577642b496549e3ba56d499b607abac)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:50.300715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:50.354041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c577642b496549e3ba56d499b607abac'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:50.354719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c577642b496549e3ba56d499b607abac'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:50.363390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c577642b496549e3ba56d499b607abac'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:50.364225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c577642b496549e3ba56d499b607abac'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:50.372530 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c577642b496549e3ba56d499b607abac'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:50.390705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1121/2260] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:06:50 2022] DELETE /identity/v3/users/c577642b496549e3ba56d499b607abac => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:50.395617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6789f-3f27-4813-8121-856cf9051679 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:50.396111 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6789f-3f27-4813-8121-856cf9051679 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:50.396111 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6789f-3f27-4813-8121-856cf9051679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:50.415696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-38d6789f-3f27-4813-8121-856cf9051679 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:50.416921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-38d6789f-3f27-4813-8121-856cf9051679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:50.479145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1140/2261] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:06:50.489079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:50.490708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:50.553411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded78542-1b8a-446e-9404-d694b08cdf3f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:50.557659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae75e9-83b3-4e45-8dad-f29fdaa09616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:50.558351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae75e9-83b3-4e45-8dad-f29fdaa09616 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:50.558637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae75e9-83b3-4e45-8dad-f29fdaa09616 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:50.558912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae75e9-83b3-4e45-8dad-f29fdaa09616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:50.569779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae75e9-83b3-4e45-8dad-f29fdaa09616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:50.571233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae75e9-83b3-4e45-8dad-f29fdaa09616 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:50.581837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1122/2262] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:06:50.915754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6789f-3f27-4813-8121-856cf9051679 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:50.917452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-38d6789f-3f27-4813-8121-856cf9051679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:50.933450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6789f-3f27-4813-8121-856cf9051679 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:50.938363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:50.939284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:50.939565 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:50.939839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] PATH_INFO: `/v3/projects/52084318387c42afb69455b94bbd3b50` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:50.941055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=52084318387c42afb69455b94bbd3b50)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:50.943609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:50.999701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52084318387c42afb69455b94bbd3b50', 'request_id': 'req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.009393 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52084318387c42afb69455b94bbd3b50', 'request_id': 'req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.015363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52084318387c42afb69455b94bbd3b50', 'request_id': 'req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.027229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] The token cache is being invalidate because project 52084318387c42afb69455b94bbd3b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:06:51.027643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.028386 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.031564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1141/2263] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:06:50 2022] DELETE /identity/v3/projects/52084318387c42afb69455b94bbd3b50 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:51.034560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae75e9-83b3-4e45-8dad-f29fdaa09616 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:51.044821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae75e9-83b3-4e45-8dad-f29fdaa09616 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:51.063099 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:51.107478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:51.108421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:51.108702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:51.108997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] PATH_INFO: `/v3/users/eaaa0f5a0c4e4028add0e28f7207380f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:51.110253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaaa0f5a0c4e4028add0e28f7207380f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:51.112187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:51.162166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaaa0f5a0c4e4028add0e28f7207380f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.162514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaaa0f5a0c4e4028add0e28f7207380f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.171738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaaa0f5a0c4e4028add0e28f7207380f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.172537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaaa0f5a0c4e4028add0e28f7207380f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.179801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-6fd8e857-af8b-4aac-8a42-accb13bba6e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaaa0f5a0c4e4028add0e28f7207380f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.200699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1123/2264] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:06:51 2022] DELETE /identity/v3/users/eaaa0f5a0c4e4028add0e28f7207380f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:51.494389 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:51.495985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:51.512744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86898d-75ab-48e1-b1ad-bdbe2f881fab admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:51.572942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:51.573876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:51.574183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:51.574462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] PATH_INFO: `/v3/projects/234666e90dab440cb969534b73491ca5` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:51.575669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=234666e90dab440cb969534b73491ca5)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:51.577598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:51.612801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:51.635264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234666e90dab440cb969534b73491ca5', 'request_id': 'req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.644762 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234666e90dab440cb969534b73491ca5', 'request_id': 'req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.651290 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234666e90dab440cb969534b73491ca5', 'request_id': 'req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.664073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] The token cache is being invalidate because project 234666e90dab440cb969534b73491ca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:06:51.664369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.665042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-84f76c76-f5fc-469d-b447-ba7b2fc2d7a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:51.666725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1142/2265] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:06:51 2022] DELETE /identity/v3/projects/234666e90dab440cb969534b73491ca5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:51.672629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8c8d2-bc8a-443a-8f35-dfd6aae4786f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:51.673140 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8c8d2-bc8a-443a-8f35-dfd6aae4786f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:51.673235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8c8d2-bc8a-443a-8f35-dfd6aae4786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:51.695860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-87e8c8d2-bc8a-443a-8f35-dfd6aae4786f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:51.697294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-87e8c8d2-bc8a-443a-8f35-dfd6aae4786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:51.777428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1124/2266] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:51.786194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deb062-0536-4415-86f9-3369c91b378c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:51.786434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deb062-0536-4415-86f9-3369c91b378c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:51.786709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deb062-0536-4415-86f9-3369c91b378c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:51.806126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b6deb062-0536-4415-86f9-3369c91b378c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:51.807360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b6deb062-0536-4415-86f9-3369c91b378c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:51.817268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1143/2267] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:51.822662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8c8d2-bc8a-443a-8f35-dfd6aae4786f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:51.838164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8c8d2-bc8a-443a-8f35-dfd6aae4786f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:51.847126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bccbb0-7660-4a21-8548-314fa94f3e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:51.848147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bccbb0-7660-4a21-8548-314fa94f3e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:51.848515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bccbb0-7660-4a21-8548-314fa94f3e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:51.848842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bccbb0-7660-4a21-8548-314fa94f3e9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:51.849632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bccbb0-7660-4a21-8548-314fa94f3e9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:06:51.849947 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bccbb0-7660-4a21-8548-314fa94f3e9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:51.851630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bccbb0-7660-4a21-8548-314fa94f3e9d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:51.864217 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1125/2268] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:06:51.868800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deb062-0536-4415-86f9-3369c91b378c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:51.878605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deb062-0536-4415-86f9-3369c91b378c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:51.884281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26325dd-6fff-4f91-a0da-b80825b0ee6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:51.884956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26325dd-6fff-4f91-a0da-b80825b0ee6f admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:51.885257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26325dd-6fff-4f91-a0da-b80825b0ee6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:51.885532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26325dd-6fff-4f91-a0da-b80825b0ee6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:51.886305 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26325dd-6fff-4f91-a0da-b80825b0ee6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:51.888238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26325dd-6fff-4f91-a0da-b80825b0ee6f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:51.934721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1144/2269] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:06:51 2022] POST /identity/v3/projects => generated 363 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:51.939508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bccbb0-7660-4a21-8548-314fa94f3e9d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:51.950816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bccbb0-7660-4a21-8548-314fa94f3e9d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:51.980132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:52.019372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327cf92-de25-44a8-b872-4c614cff1b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.020154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327cf92-de25-44a8-b872-4c614cff1b14 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.020505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327cf92-de25-44a8-b872-4c614cff1b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.020774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327cf92-de25-44a8-b872-4c614cff1b14 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.021524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327cf92-de25-44a8-b872-4c614cff1b14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.024070 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327cf92-de25-44a8-b872-4c614cff1b14 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.046178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1126/2270] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:06:51 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:52.051604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26325dd-6fff-4f91-a0da-b80825b0ee6f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:52.062732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26325dd-6fff-4f91-a0da-b80825b0ee6f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:52.067871 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a3d3a-91aa-4350-a409-5f7924a76ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.069578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a3d3a-91aa-4350-a409-5f7924a76ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.069861 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a3d3a-91aa-4350-a409-5f7924a76ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.070212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a3d3a-91aa-4350-a409-5f7924a76ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.070914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a3d3a-91aa-4350-a409-5f7924a76ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.072549 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a3d3a-91aa-4350-a409-5f7924a76ae2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.083090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1145/2271] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:52.087495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327cf92-de25-44a8-b872-4c614cff1b14 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:52.098072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327cf92-de25-44a8-b872-4c614cff1b14 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:52.103758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551cab4-37f4-420b-a6c6-f3b948ad7cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.104426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551cab4-37f4-420b-a6c6-f3b948ad7cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.104709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551cab4-37f4-420b-a6c6-f3b948ad7cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.104988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551cab4-37f4-420b-a6c6-f3b948ad7cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.105730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551cab4-37f4-420b-a6c6-f3b948ad7cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.107400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551cab4-37f4-420b-a6c6-f3b948ad7cb2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.118911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1127/2272] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:52.123291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a3d3a-91aa-4350-a409-5f7924a76ae2 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:52.134187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a3d3a-91aa-4350-a409-5f7924a76ae2 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:52.139206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26970751-8acd-4d2a-9a4b-6d522083e8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.140089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970751-8acd-4d2a-9a4b-6d522083e8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.140357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970751-8acd-4d2a-9a4b-6d522083e8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.140622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26970751-8acd-4d2a-9a4b-6d522083e8f1 admin admin] PATH_INFO: `/v3/projects/d949d6ccedf24140902ebd46ed8dc937/users/b311c112c84b4e1695c5b7e227f893a3/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.158387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26970751-8acd-4d2a-9a4b-6d522083e8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d949d6ccedf24140902ebd46ed8dc937, user_id=b311c112c84b4e1695c5b7e227f893a3, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.161138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26970751-8acd-4d2a-9a4b-6d522083e8f1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.175049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1146/2273] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:06:52 2022] PUT /identity/v3/projects/d949d6ccedf24140902ebd46ed8dc937/users/b311c112c84b4e1695c5b7e227f893a3/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:52.189010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e41ad-5abc-4861-b339-68ea13fd891c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.189301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e41ad-5abc-4861-b339-68ea13fd891c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.189633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e41ad-5abc-4861-b339-68ea13fd891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.221341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-af9e41ad-5abc-4861-b339-68ea13fd891c None None] MFA Rules not processed for user `b311c112c84b4e1695c5b7e227f893a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:52.223922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-af9e41ad-5abc-4861-b339-68ea13fd891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:52.297727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1128/2274] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:52 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:52.302431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26970751-8acd-4d2a-9a4b-6d522083e8f1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:52.374895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26970751-8acd-4d2a-9a4b-6d522083e8f1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:52.380321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f2519-db30-4b8d-bfd2-4685a80c9582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.381075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f2519-db30-4b8d-bfd2-4685a80c9582 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.381851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f2519-db30-4b8d-bfd2-4685a80c9582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.382183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f2519-db30-4b8d-bfd2-4685a80c9582 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.382893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f2519-db30-4b8d-bfd2-4685a80c9582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.384840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f2519-db30-4b8d-bfd2-4685a80c9582 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.419363 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1147/2275] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:06:52 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:06:52.420565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e41ad-5abc-4861-b339-68ea13fd891c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:52.432424 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e41ad-5abc-4861-b339-68ea13fd891c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:52.491392 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba70d64-5f22-4b9c-9a86-6339fe0acd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.492071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba70d64-5f22-4b9c-9a86-6339fe0acd7a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.492349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba70d64-5f22-4b9c-9a86-6339fe0acd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.492622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba70d64-5f22-4b9c-9a86-6339fe0acd7a admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.493325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba70d64-5f22-4b9c-9a86-6339fe0acd7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.495328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba70d64-5f22-4b9c-9a86-6339fe0acd7a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.516876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1129/2276] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:06:52 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:06:52.521072 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f2519-db30-4b8d-bfd2-4685a80c9582 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:52.531855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f2519-db30-4b8d-bfd2-4685a80c9582 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:52.536797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103280b1-7fa2-4cdb-b758-ebf4ec6c92ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.537555 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103280b1-7fa2-4cdb-b758-ebf4ec6c92ed admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.537837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103280b1-7fa2-4cdb-b758-ebf4ec6c92ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.538139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103280b1-7fa2-4cdb-b758-ebf4ec6c92ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.538893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103280b1-7fa2-4cdb-b758-ebf4ec6c92ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.540490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103280b1-7fa2-4cdb-b758-ebf4ec6c92ed admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.551777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1148/2277] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:52.556453 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba70d64-5f22-4b9c-9a86-6339fe0acd7a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:52.566563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba70d64-5f22-4b9c-9a86-6339fe0acd7a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:52.571770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa4ac0-c332-4ed3-9bce-9a9535291ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.572655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa4ac0-c332-4ed3-9bce-9a9535291ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.572932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa4ac0-c332-4ed3-9bce-9a9535291ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.573232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa4ac0-c332-4ed3-9bce-9a9535291ce5 admin admin] PATH_INFO: `/v3/projects/22a5bce8837041b4a227d08a7ed6dd07/users/c225fee748c94ab0b060852b1a59aa87/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.588409 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa4ac0-c332-4ed3-9bce-9a9535291ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a5bce8837041b4a227d08a7ed6dd07, user_id=c225fee748c94ab0b060852b1a59aa87, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.591125 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa4ac0-c332-4ed3-9bce-9a9535291ce5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.605562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1130/2278] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:06:52 2022] PUT /identity/v3/projects/22a5bce8837041b4a227d08a7ed6dd07/users/c225fee748c94ab0b060852b1a59aa87/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:06:52.609675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103280b1-7fa2-4cdb-b758-ebf4ec6c92ed admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:52.620785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103280b1-7fa2-4cdb-b758-ebf4ec6c92ed admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:52.641570 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:52.679770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37785b7-9f6b-453e-9720-c16d5ee52db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.680443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37785b7-9f6b-453e-9720-c16d5ee52db0 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.680729 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37785b7-9f6b-453e-9720-c16d5ee52db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.681007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37785b7-9f6b-453e-9720-c16d5ee52db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.681740 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37785b7-9f6b-453e-9720-c16d5ee52db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.683471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37785b7-9f6b-453e-9720-c16d5ee52db0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.694627 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1149/2279] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:52.699186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa4ac0-c332-4ed3-9bce-9a9535291ce5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:52.704919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:52.711637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa4ac0-c332-4ed3-9bce-9a9535291ce5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:52.720446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41a461-9678-4fda-a404-34a186b19b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.721867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41a461-9678-4fda-a404-34a186b19b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.722301 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41a461-9678-4fda-a404-34a186b19b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.722690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41a461-9678-4fda-a404-34a186b19b84 admin admin] PATH_INFO: `/v3/projects/22a5bce8837041b4a227d08a7ed6dd07/users/c225fee748c94ab0b060852b1a59aa87/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.725765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41a461-9678-4fda-a404-34a186b19b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a5bce8837041b4a227d08a7ed6dd07, user_id=c225fee748c94ab0b060852b1a59aa87, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.728575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41a461-9678-4fda-a404-34a186b19b84 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.739954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1131/2280] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:06:52 2022] PUT /identity/v3/projects/22a5bce8837041b4a227d08a7ed6dd07/users/c225fee748c94ab0b060852b1a59aa87/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:52.750488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d10ee-6210-48bc-bdfd-01a19faec712 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.750777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d10ee-6210-48bc-bdfd-01a19faec712 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.751058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d10ee-6210-48bc-bdfd-01a19faec712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.779693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-888d10ee-6210-48bc-bdfd-01a19faec712 None None] MFA Rules not processed for user `c225fee748c94ab0b060852b1a59aa87`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:52.780900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-888d10ee-6210-48bc-bdfd-01a19faec712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:52.855402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1150/2281] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:52 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:52.866586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41a461-9678-4fda-a404-34a186b19b84 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:52.868748 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2d41a461-9678-4fda-a404-34a186b19b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:52.933689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41a461-9678-4fda-a404-34a186b19b84 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:52.937540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012ac2c-e06b-4296-a2e0-52e65da27c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:52.938206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012ac2c-e06b-4296-a2e0-52e65da27c2e service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:52.938516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012ac2c-e06b-4296-a2e0-52e65da27c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:52.938789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012ac2c-e06b-4296-a2e0-52e65da27c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:52.948761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012ac2c-e06b-4296-a2e0-52e65da27c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:52.950242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012ac2c-e06b-4296-a2e0-52e65da27c2e service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:52.960088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1132/2282] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:06:52 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:06:54.430733 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d10ee-6210-48bc-bdfd-01a19faec712 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:54.501173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d10ee-6210-48bc-bdfd-01a19faec712 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:54.506001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:54.507036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:54.507468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:54.507745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] PATH_INFO: `/v3/users/b311c112c84b4e1695c5b7e227f893a3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:54.509011 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b311c112c84b4e1695c5b7e227f893a3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:54.511065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:54.562387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b311c112c84b4e1695c5b7e227f893a3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:54.564178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:54.574928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b311c112c84b4e1695c5b7e227f893a3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:54.576008 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b311c112c84b4e1695c5b7e227f893a3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:54.576691 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b311c112c84b4e1695c5b7e227f893a3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:54.589685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b311c112c84b4e1695c5b7e227f893a3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:54.601959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1151/2283] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:06:54 2022] DELETE /identity/v3/users/b311c112c84b4e1695c5b7e227f893a3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:54.607206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ad72f-7420-4d38-9e9c-ac63b04091c2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:54.607674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ad72f-7420-4d38-9e9c-ac63b04091c2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:54.607790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ad72f-7420-4d38-9e9c-ac63b04091c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:54.625872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-202ad72f-7420-4d38-9e9c-ac63b04091c2 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:54.627140 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-202ad72f-7420-4d38-9e9c-ac63b04091c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:54.689471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1133/2284] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:54.700490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:54.702959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:54.765696 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c4670-0883-4a57-bcb3-9e0c6f0e7d19 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:54.769714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270eb7b-a43a-444c-a710-b31afb974375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:54.770391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270eb7b-a43a-444c-a710-b31afb974375 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:54.770671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270eb7b-a43a-444c-a710-b31afb974375 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:54.770947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270eb7b-a43a-444c-a710-b31afb974375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:54.780926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270eb7b-a43a-444c-a710-b31afb974375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:54.782435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270eb7b-a43a-444c-a710-b31afb974375 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:54.792461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1152/2285] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:06:55.071137 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ad72f-7420-4d38-9e9c-ac63b04091c2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:55.072730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-202ad72f-7420-4d38-9e9c-ac63b04091c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:55.160331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ad72f-7420-4d38-9e9c-ac63b04091c2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:55.165948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:55.166847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:55.167128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:55.167402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] PATH_INFO: `/v3/projects/d949d6ccedf24140902ebd46ed8dc937` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:55.168596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d949d6ccedf24140902ebd46ed8dc937)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:55.170552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:55.204703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:55.221868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd949d6ccedf24140902ebd46ed8dc937', 'request_id': 'req-d2671886-7a7e-4945-8822-31ebd9526d08'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.231946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd949d6ccedf24140902ebd46ed8dc937', 'request_id': 'req-d2671886-7a7e-4945-8822-31ebd9526d08'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.237621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd949d6ccedf24140902ebd46ed8dc937', 'request_id': 'req-d2671886-7a7e-4945-8822-31ebd9526d08'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.249564 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] The token cache is being invalidate because project d949d6ccedf24140902ebd46ed8dc937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:06:55.249847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.250532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.252168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1134/2286] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:06:55 2022] DELETE /identity/v3/projects/d949d6ccedf24140902ebd46ed8dc937 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:55.256147 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270eb7b-a43a-444c-a710-b31afb974375 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:55.309597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270eb7b-a43a-444c-a710-b31afb974375 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:55.368178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:55.369064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:55.369345 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:55.369619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] PATH_INFO: `/v3/users/c225fee748c94ab0b060852b1a59aa87` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:55.370867 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c225fee748c94ab0b060852b1a59aa87)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:55.372818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:55.420624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:INSERT=1,keystone:DELETE=8,keystone:SELECT=91 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:55.426397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c225fee748c94ab0b060852b1a59aa87'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.435573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c225fee748c94ab0b060852b1a59aa87'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.437217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c225fee748c94ab0b060852b1a59aa87'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.437932 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c225fee748c94ab0b060852b1a59aa87'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.454277 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6af28689-0e27-4658-890f-615a0c0e8b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c225fee748c94ab0b060852b1a59aa87'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.467606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1153/2287] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:06:55 2022] DELETE /identity/v3/users/c225fee748c94ab0b060852b1a59aa87 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:06:55.746936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:55.748441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:55.761986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2671886-7a7e-4945-8822-31ebd9526d08 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:55.822373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:55.823175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:55.823454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:55.823730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] PATH_INFO: `/v3/projects/22a5bce8837041b4a227d08a7ed6dd07` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:55.824930 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a5bce8837041b4a227d08a7ed6dd07)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:55.826922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:55.884575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a5bce8837041b4a227d08a7ed6dd07', 'request_id': 'req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.892724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a5bce8837041b4a227d08a7ed6dd07', 'request_id': 'req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.898568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a5bce8837041b4a227d08a7ed6dd07', 'request_id': 'req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.911143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] The token cache is being invalidate because project 22a5bce8837041b4a227d08a7ed6dd07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:06:55.911584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.912294 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4a761643-76a7-41cd-9eb1-d8cd56d4ae37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:06:55.914158 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1135/2288] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:06:55 2022] DELETE /identity/v3/projects/22a5bce8837041b4a227d08a7ed6dd07 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:55.919173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e66d1e-907a-4d84-9ac3-5c9093d11282 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:55.919458 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e66d1e-907a-4d84-9ac3-5c9093d11282 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:55.919732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e66d1e-907a-4d84-9ac3-5c9093d11282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:55.971102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-89e66d1e-907a-4d84-9ac3-5c9093d11282 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:55.972321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-89e66d1e-907a-4d84-9ac3-5c9093d11282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:56.032543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1154/2289] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:56.041307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8b421-43f8-4408-86a7-9e1d53af70c0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.041660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8b421-43f8-4408-86a7-9e1d53af70c0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.042044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8b421-43f8-4408-86a7-9e1d53af70c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.066262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a8d8b421-43f8-4408-86a7-9e1d53af70c0 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:56.067472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a8d8b421-43f8-4408-86a7-9e1d53af70c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:56.076779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1136/2290] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:06:56.082300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e66d1e-907a-4d84-9ac3-5c9093d11282 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.095033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e66d1e-907a-4d84-9ac3-5c9093d11282 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.100788 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7199ae1-3ebf-4abe-9615-58d0a021fc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.101484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7199ae1-3ebf-4abe-9615-58d0a021fc30 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.101768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7199ae1-3ebf-4abe-9615-58d0a021fc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.102036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7199ae1-3ebf-4abe-9615-58d0a021fc30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.103560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7199ae1-3ebf-4abe-9615-58d0a021fc30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:06:56.103902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7199ae1-3ebf-4abe-9615-58d0a021fc30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.105466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7199ae1-3ebf-4abe-9615-58d0a021fc30 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.119339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1155/2291] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:06:56.124323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8b421-43f8-4408-86a7-9e1d53af70c0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.133878 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8b421-43f8-4408-86a7-9e1d53af70c0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.139014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6174b-56ed-4676-8acd-c6b8e7282a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.139764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6174b-56ed-4676-8acd-c6b8e7282a1d admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.140127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6174b-56ed-4676-8acd-c6b8e7282a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.140472 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6174b-56ed-4676-8acd-c6b8e7282a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.141241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6174b-56ed-4676-8acd-c6b8e7282a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.143280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6174b-56ed-4676-8acd-c6b8e7282a1d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.176630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1137/2292] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:06:56 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:56.181522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7199ae1-3ebf-4abe-9615-58d0a021fc30 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.191848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7199ae1-3ebf-4abe-9615-58d0a021fc30 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.249931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ee471-6094-4eb8-817b-a4c01c1544e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.250629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee471-6094-4eb8-817b-a4c01c1544e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.250910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee471-6094-4eb8-817b-a4c01c1544e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.251179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee471-6094-4eb8-817b-a4c01c1544e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.251863 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ee471-6094-4eb8-817b-a4c01c1544e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.253828 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ee471-6094-4eb8-817b-a4c01c1544e2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.276948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1156/2293] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:06:56 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:56.281155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6174b-56ed-4676-8acd-c6b8e7282a1d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.292120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6174b-56ed-4676-8acd-c6b8e7282a1d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.298075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700316b8-d56f-4e43-a186-f819a90a26cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.298868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700316b8-d56f-4e43-a186-f819a90a26cf admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.299151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700316b8-d56f-4e43-a186-f819a90a26cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.299429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700316b8-d56f-4e43-a186-f819a90a26cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.300212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700316b8-d56f-4e43-a186-f819a90a26cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.301850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700316b8-d56f-4e43-a186-f819a90a26cf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.313442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1138/2294] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:56.317702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ee471-6094-4eb8-817b-a4c01c1544e2 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.329399 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ee471-6094-4eb8-817b-a4c01c1544e2 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.335264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcd215-e19e-4acd-9d66-a10f39f9c6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.335938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcd215-e19e-4acd-9d66-a10f39f9c6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.336222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcd215-e19e-4acd-9d66-a10f39f9c6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.336503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcd215-e19e-4acd-9d66-a10f39f9c6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.337243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcd215-e19e-4acd-9d66-a10f39f9c6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.338955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcd215-e19e-4acd-9d66-a10f39f9c6c7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.341221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:56.352383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1157/2295] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:56.356450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700316b8-d56f-4e43-a186-f819a90a26cf admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.362925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:56.367278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700316b8-d56f-4e43-a186-f819a90a26cf admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.374048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ecc03-41cc-4baf-9242-549946bad27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.374881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ecc03-41cc-4baf-9242-549946bad27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.375166 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ecc03-41cc-4baf-9242-549946bad27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.375459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ecc03-41cc-4baf-9242-549946bad27f admin admin] PATH_INFO: `/v3/projects/92ddfb37b949419f9f0b6b5df21aeb41/users/f4e1dd14609641a0b69f99bf92342886/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.392697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ecc03-41cc-4baf-9242-549946bad27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ddfb37b949419f9f0b6b5df21aeb41, user_id=f4e1dd14609641a0b69f99bf92342886, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.395820 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ecc03-41cc-4baf-9242-549946bad27f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.410157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1139/2296] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:06:56 2022] PUT /identity/v3/projects/92ddfb37b949419f9f0b6b5df21aeb41/users/f4e1dd14609641a0b69f99bf92342886/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:56.420977 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd90323-d2bb-4d4a-babb-ad2bb02eff2f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.421269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd90323-d2bb-4d4a-babb-ad2bb02eff2f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.421536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd90323-d2bb-4d4a-babb-ad2bb02eff2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.452725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5fd90323-d2bb-4d4a-babb-ad2bb02eff2f None None] MFA Rules not processed for user `f4e1dd14609641a0b69f99bf92342886`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:56.453931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5fd90323-d2bb-4d4a-babb-ad2bb02eff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:56.519024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1158/2297] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:06:56.523734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ecc03-41cc-4baf-9242-549946bad27f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.592630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ecc03-41cc-4baf-9242-549946bad27f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.598621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1835853-d069-4573-a63c-26917846864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.599403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1835853-d069-4573-a63c-26917846864c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.599686 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1835853-d069-4573-a63c-26917846864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.599982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1835853-d069-4573-a63c-26917846864c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.600677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1835853-d069-4573-a63c-26917846864c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.602635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1835853-d069-4573-a63c-26917846864c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.631763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1140/2298] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:06:56 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:56.635646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd90323-d2bb-4d4a-babb-ad2bb02eff2f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.646631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd90323-d2bb-4d4a-babb-ad2bb02eff2f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.716875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d17ae-07d6-4d03-b64f-596adde9133f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.717678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d17ae-07d6-4d03-b64f-596adde9133f admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.718028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d17ae-07d6-4d03-b64f-596adde9133f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.718416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d17ae-07d6-4d03-b64f-596adde9133f admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.719400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d17ae-07d6-4d03-b64f-596adde9133f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.721700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d17ae-07d6-4d03-b64f-596adde9133f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.745131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1159/2299] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:06:56 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:06:56.746371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1835853-d069-4573-a63c-26917846864c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.756594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1835853-d069-4573-a63c-26917846864c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.761591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a71c7e-1408-423d-891c-11d93a9f4c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.762289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a71c7e-1408-423d-891c-11d93a9f4c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.762570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a71c7e-1408-423d-891c-11d93a9f4c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.762843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a71c7e-1408-423d-891c-11d93a9f4c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.763577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a71c7e-1408-423d-891c-11d93a9f4c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.765193 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a71c7e-1408-423d-891c-11d93a9f4c0f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.776312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1141/2300] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:56.780916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d17ae-07d6-4d03-b64f-596adde9133f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.795293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d17ae-07d6-4d03-b64f-596adde9133f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.800577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c5660-e25c-4da6-9c5a-a25464e7312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.801511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c5660-e25c-4da6-9c5a-a25464e7312a admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.801794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c5660-e25c-4da6-9c5a-a25464e7312a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.802120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c5660-e25c-4da6-9c5a-a25464e7312a admin admin] PATH_INFO: `/v3/projects/0abe0b407dd44f44a26860388bb2f3c8/users/7218c4676dcd4f57a05dab0dc61cbed9/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.817513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c5660-e25c-4da6-9c5a-a25464e7312a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0abe0b407dd44f44a26860388bb2f3c8, user_id=7218c4676dcd4f57a05dab0dc61cbed9, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.820580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c5660-e25c-4da6-9c5a-a25464e7312a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.835968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1160/2301] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:06:56 2022] PUT /identity/v3/projects/0abe0b407dd44f44a26860388bb2f3c8/users/7218c4676dcd4f57a05dab0dc61cbed9/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:56.840450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a71c7e-1408-423d-891c-11d93a9f4c0f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.850345 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a71c7e-1408-423d-891c-11d93a9f4c0f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.909987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab6da5-2884-4009-b105-cffe2ca1f654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.910699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab6da5-2884-4009-b105-cffe2ca1f654 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.911528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab6da5-2884-4009-b105-cffe2ca1f654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.911528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab6da5-2884-4009-b105-cffe2ca1f654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.912180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab6da5-2884-4009-b105-cffe2ca1f654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.913945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab6da5-2884-4009-b105-cffe2ca1f654 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.924993 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1142/2302] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:06:56.929953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c5660-e25c-4da6-9c5a-a25464e7312a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:56.941545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c5660-e25c-4da6-9c5a-a25464e7312a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:56.947852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b8869-42bc-4587-afb1-127d7236749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:56.948894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b8869-42bc-4587-afb1-127d7236749f admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.949178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b8869-42bc-4587-afb1-127d7236749f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.949453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b8869-42bc-4587-afb1-127d7236749f admin admin] PATH_INFO: `/v3/projects/0abe0b407dd44f44a26860388bb2f3c8/users/7218c4676dcd4f57a05dab0dc61cbed9/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:56.951926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b8869-42bc-4587-afb1-127d7236749f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0abe0b407dd44f44a26860388bb2f3c8, user_id=7218c4676dcd4f57a05dab0dc61cbed9, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:56.954647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b8869-42bc-4587-afb1-127d7236749f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:56.966695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1161/2303] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:06:56 2022] PUT /identity/v3/projects/0abe0b407dd44f44a26860388bb2f3c8/users/7218c4676dcd4f57a05dab0dc61cbed9/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:06:56.978108 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee2a71-3da4-4f30-a5ee-12220a711c2c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:56.978468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee2a71-3da4-4f30-a5ee-12220a711c2c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:56.978766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee2a71-3da4-4f30-a5ee-12220a711c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:57.013147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-56ee2a71-3da4-4f30-a5ee-12220a711c2c None None] MFA Rules not processed for user `7218c4676dcd4f57a05dab0dc61cbed9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:06:57.014419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-56ee2a71-3da4-4f30-a5ee-12220a711c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:57.059979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:57.090423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1143/2304] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:06:57.101994 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b8869-42bc-4587-afb1-127d7236749f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:57.104712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d24b8869-42bc-4587-afb1-127d7236749f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:57.153718 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:06:57.188514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b8869-42bc-4587-afb1-127d7236749f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:57.192365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38622a3b-35ee-4fa1-aa67-20f31c7118b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:57.193367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38622a3b-35ee-4fa1-aa67-20f31c7118b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:57.193741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38622a3b-35ee-4fa1-aa67-20f31c7118b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:57.194142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38622a3b-35ee-4fa1-aa67-20f31c7118b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:57.204940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38622a3b-35ee-4fa1-aa67-20f31c7118b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:57.206426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38622a3b-35ee-4fa1-aa67-20f31c7118b9 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:57.216331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1162/2305] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:06:57 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:06:57.265978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee2a71-3da4-4f30-a5ee-12220a711c2c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:06:57.267476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-56ee2a71-3da4-4f30-a5ee-12220a711c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:06:57.280533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee2a71-3da4-4f30-a5ee-12220a711c2c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:06:57.284697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e61e9-ff0a-4884-bbc2-947f1eb47fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:06:57.285353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e61e9-ff0a-4884-bbc2-947f1eb47fda service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:06:57.285646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e61e9-ff0a-4884-bbc2-947f1eb47fda service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:06:57.285920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e61e9-ff0a-4884-bbc2-947f1eb47fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:06:57.351805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e61e9-ff0a-4884-bbc2-947f1eb47fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:06:57.353861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e61e9-ff0a-4884-bbc2-947f1eb47fda service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:06:57.363640 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1144/2306] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:06:57 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:00.730849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38622a3b-35ee-4fa1-aa67-20f31c7118b9 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:00.795201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38622a3b-35ee-4fa1-aa67-20f31c7118b9 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:00.799902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:00.800776 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:00.801053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:00.801321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] PATH_INFO: `/v3/users/f4e1dd14609641a0b69f99bf92342886` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:00.802526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e1dd14609641a0b69f99bf92342886)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:00.804449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:00.856137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e1dd14609641a0b69f99bf92342886'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:00.864705 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e1dd14609641a0b69f99bf92342886'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:00.865426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e1dd14609641a0b69f99bf92342886'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:00.866040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e1dd14609641a0b69f99bf92342886'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:00.878469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e1dd14609641a0b69f99bf92342886'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:00.890672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1163/2307] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:00 2022] DELETE /identity/v3/users/f4e1dd14609641a0b69f99bf92342886 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:00.895254 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e663131-e765-4465-8f95-a2851f9f05ac None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:00.895760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e663131-e765-4465-8f95-a2851f9f05ac None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:00.895760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e663131-e765-4465-8f95-a2851f9f05ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:00.913908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0e663131-e765-4465-8f95-a2851f9f05ac None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:00.915160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0e663131-e765-4465-8f95-a2851f9f05ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:00.978299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1145/2308] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:00.988093 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:00.989578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:01.043421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:01.077477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29d47f-c5b8-4b5b-8aed-c668cdda2cbc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:01.081457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46390ad0-000a-46fe-822f-fc2c12eebcdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:01.082132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46390ad0-000a-46fe-822f-fc2c12eebcdb service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:01.082408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46390ad0-000a-46fe-822f-fc2c12eebcdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:01.082682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46390ad0-000a-46fe-822f-fc2c12eebcdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:01.092352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46390ad0-000a-46fe-822f-fc2c12eebcdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:01.093789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46390ad0-000a-46fe-822f-fc2c12eebcdb service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:01.103960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1164/2309] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:01.461345 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e663131-e765-4465-8f95-a2851f9f05ac None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:01.463972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0e663131-e765-4465-8f95-a2851f9f05ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:01.475984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e663131-e765-4465-8f95-a2851f9f05ac None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:01.481565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:01.482500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:01.482778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:01.483049 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] PATH_INFO: `/v3/projects/92ddfb37b949419f9f0b6b5df21aeb41` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:01.484387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ddfb37b949419f9f0b6b5df21aeb41)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:01.486349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:01.551556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ddfb37b949419f9f0b6b5df21aeb41', 'request_id': 'req-3707ff9e-173d-48c3-9879-f12649177e6a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:01.560011 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ddfb37b949419f9f0b6b5df21aeb41', 'request_id': 'req-3707ff9e-173d-48c3-9879-f12649177e6a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:01.565542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ddfb37b949419f9f0b6b5df21aeb41', 'request_id': 'req-3707ff9e-173d-48c3-9879-f12649177e6a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:01.573168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:01.579001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] The token cache is being invalidate because project 92ddfb37b949419f9f0b6b5df21aeb41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:01.579565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:01.580369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:01.582449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1146/2310] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:01 2022] DELETE /identity/v3/projects/92ddfb37b949419f9f0b6b5df21aeb41 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:01.586747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46390ad0-000a-46fe-822f-fc2c12eebcdb service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:01.597007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46390ad0-000a-46fe-822f-fc2c12eebcdb service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:01.659258 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:01.660152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:01.660446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:01.660727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] PATH_INFO: `/v3/users/7218c4676dcd4f57a05dab0dc61cbed9` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:01.661916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7218c4676dcd4f57a05dab0dc61cbed9)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:01.663865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:01.721202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7218c4676dcd4f57a05dab0dc61cbed9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:01.730215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7218c4676dcd4f57a05dab0dc61cbed9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:01.731266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7218c4676dcd4f57a05dab0dc61cbed9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:01.732220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7218c4676dcd4f57a05dab0dc61cbed9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:01.750378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-934ea6c6-f9e0-432f-824f-8233e885d8a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7218c4676dcd4f57a05dab0dc61cbed9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:01.766306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1165/2311] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:01 2022] DELETE /identity/v3/users/7218c4676dcd4f57a05dab0dc61cbed9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:07:02.069421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:02.071507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:02.086213 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707ff9e-173d-48c3-9879-f12649177e6a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:02.144680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:02.145547 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.145827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.146132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] PATH_INFO: `/v3/projects/0abe0b407dd44f44a26860388bb2f3c8` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.147299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=0abe0b407dd44f44a26860388bb2f3c8)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:02.149233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:02.198024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0abe0b407dd44f44a26860388bb2f3c8', 'request_id': 'req-f1d466b4-9714-4900-9745-255a4cfd9ced'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:02.206517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0abe0b407dd44f44a26860388bb2f3c8', 'request_id': 'req-f1d466b4-9714-4900-9745-255a4cfd9ced'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:02.212186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0abe0b407dd44f44a26860388bb2f3c8', 'request_id': 'req-f1d466b4-9714-4900-9745-255a4cfd9ced'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:02.223212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] The token cache is being invalidate because project 0abe0b407dd44f44a26860388bb2f3c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:02.223500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:02.224167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f1d466b4-9714-4900-9745-255a4cfd9ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:02.225747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1147/2312] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:02 2022] DELETE /identity/v3/projects/0abe0b407dd44f44a26860388bb2f3c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:02.231361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c290b79-d26c-44e1-8795-eff917b5d861 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.231649 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c290b79-d26c-44e1-8795-eff917b5d861 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.231928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c290b79-d26c-44e1-8795-eff917b5d861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.249758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2c290b79-d26c-44e1-8795-eff917b5d861 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:02.250945 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2c290b79-d26c-44e1-8795-eff917b5d861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:02.296206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:02.313587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1166/2313] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:02.322116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03880f2-782b-438d-a0e1-fb87b1b0cd49 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.322501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03880f2-782b-438d-a0e1-fb87b1b0cd49 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.322801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03880f2-782b-438d-a0e1-fb87b1b0cd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.349312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b03880f2-782b-438d-a0e1-fb87b1b0cd49 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:02.351188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b03880f2-782b-438d-a0e1-fb87b1b0cd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:02.360607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1148/2314] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:02.364993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c290b79-d26c-44e1-8795-eff917b5d861 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:02.375416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c290b79-d26c-44e1-8795-eff917b5d861 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:02.381275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc6bff-0edc-4fdb-816d-3d6f413f58f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:02.381939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc6bff-0edc-4fdb-816d-3d6f413f58f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.382244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc6bff-0edc-4fdb-816d-3d6f413f58f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.382556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc6bff-0edc-4fdb-816d-3d6f413f58f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.383307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc6bff-0edc-4fdb-816d-3d6f413f58f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:07:02.383590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc6bff-0edc-4fdb-816d-3d6f413f58f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:02.385352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc6bff-0edc-4fdb-816d-3d6f413f58f9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:02.398284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1167/2315] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:07:02.403924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03880f2-782b-438d-a0e1-fb87b1b0cd49 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:02.417994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03880f2-782b-438d-a0e1-fb87b1b0cd49 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:02.423949 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca562-4371-4db2-9440-399a915a8311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:02.424599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca562-4371-4db2-9440-399a915a8311 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.426251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca562-4371-4db2-9440-399a915a8311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.426251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fca562-4371-4db2-9440-399a915a8311 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.426251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca562-4371-4db2-9440-399a915a8311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:02.427805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fca562-4371-4db2-9440-399a915a8311 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:02.464799 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1149/2316] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:02 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:02.468979 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc6bff-0edc-4fdb-816d-3d6f413f58f9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:02.479675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc6bff-0edc-4fdb-816d-3d6f413f58f9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:02.540297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664a5d0-aa91-4730-bf84-370cd6cadbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:02.540986 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664a5d0-aa91-4730-bf84-370cd6cadbc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.541381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664a5d0-aa91-4730-bf84-370cd6cadbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.541764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664a5d0-aa91-4730-bf84-370cd6cadbc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.542393 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664a5d0-aa91-4730-bf84-370cd6cadbc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:02.545885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664a5d0-aa91-4730-bf84-370cd6cadbc4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:02.575613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1168/2317] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:02 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:02.579835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca562-4371-4db2-9440-399a915a8311 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:02.593928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fca562-4371-4db2-9440-399a915a8311 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:02.599713 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca036b-3e2b-4d73-a9fd-efc4b2b5c496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:02.600604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca036b-3e2b-4d73-a9fd-efc4b2b5c496 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.600893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca036b-3e2b-4d73-a9fd-efc4b2b5c496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.601182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca036b-3e2b-4d73-a9fd-efc4b2b5c496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.601939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca036b-3e2b-4d73-a9fd-efc4b2b5c496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:02.604188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca036b-3e2b-4d73-a9fd-efc4b2b5c496 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:02.616563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1150/2318] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:02.621230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664a5d0-aa91-4730-bf84-370cd6cadbc4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:02.632690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664a5d0-aa91-4730-bf84-370cd6cadbc4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:02.638758 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fe18e-a779-4af0-a5dd-c8d2b2179799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:02.639588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fe18e-a779-4af0-a5dd-c8d2b2179799 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.639971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fe18e-a779-4af0-a5dd-c8d2b2179799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.640308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fe18e-a779-4af0-a5dd-c8d2b2179799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.641123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fe18e-a779-4af0-a5dd-c8d2b2179799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:02.642878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fe18e-a779-4af0-a5dd-c8d2b2179799 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:02.655199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1169/2319] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:02.659494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca036b-3e2b-4d73-a9fd-efc4b2b5c496 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:02.670387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca036b-3e2b-4d73-a9fd-efc4b2b5c496 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:02.675604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0a84e-dcf7-4e6d-9923-e41e4026cef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:02.676543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0a84e-dcf7-4e6d-9923-e41e4026cef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.676826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0a84e-dcf7-4e6d-9923-e41e4026cef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.677105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0a84e-dcf7-4e6d-9923-e41e4026cef1 admin admin] PATH_INFO: `/v3/projects/c75ea30ba8e14ba1aa4f3ca36d38fd39/users/900f536cc6a0423294ed37089c8e326a/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.694676 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0a84e-dcf7-4e6d-9923-e41e4026cef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75ea30ba8e14ba1aa4f3ca36d38fd39, user_id=900f536cc6a0423294ed37089c8e326a, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:02.697855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0a84e-dcf7-4e6d-9923-e41e4026cef1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:02.713869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1151/2320] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:02 2022] PUT /identity/v3/projects/c75ea30ba8e14ba1aa4f3ca36d38fd39/users/900f536cc6a0423294ed37089c8e326a/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:02.724432 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfee9c9-26bb-40b9-9032-e2032aff6032 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.724721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfee9c9-26bb-40b9-9032-e2032aff6032 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.725181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfee9c9-26bb-40b9-9032-e2032aff6032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.754534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ecfee9c9-26bb-40b9-9032-e2032aff6032 None None] MFA Rules not processed for user `900f536cc6a0423294ed37089c8e326a`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:02.755773 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ecfee9c9-26bb-40b9-9032-e2032aff6032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:02.825081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1170/2321] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:02.829617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0a84e-dcf7-4e6d-9923-e41e4026cef1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:02.857876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:02.899843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0a84e-dcf7-4e6d-9923-e41e4026cef1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:02.905644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7a339-29ee-4f23-aea3-fe52bb95683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:02.906352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7a339-29ee-4f23-aea3-fe52bb95683a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:02.906633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7a339-29ee-4f23-aea3-fe52bb95683a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:02.906907 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7a339-29ee-4f23-aea3-fe52bb95683a admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:02.907603 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7a339-29ee-4f23-aea3-fe52bb95683a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:02.909659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7a339-29ee-4f23-aea3-fe52bb95683a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:02.943780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1152/2322] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:02 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:02.948550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfee9c9-26bb-40b9-9032-e2032aff6032 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:02.961506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfee9c9-26bb-40b9-9032-e2032aff6032 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:03.005144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:03.021613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6f661-6004-44df-82a3-320fa27a3701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:03.022315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6f661-6004-44df-82a3-320fa27a3701 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:03.022669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6f661-6004-44df-82a3-320fa27a3701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:03.022946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6f661-6004-44df-82a3-320fa27a3701 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:03.023647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6f661-6004-44df-82a3-320fa27a3701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:03.025712 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6f661-6004-44df-82a3-320fa27a3701 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:03.046052 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1171/2323] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:02 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:03.050099 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7a339-29ee-4f23-aea3-fe52bb95683a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:03.060858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7a339-29ee-4f23-aea3-fe52bb95683a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:03.066149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310832c-d28a-47a2-aba6-6c19f793cfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:03.066806 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310832c-d28a-47a2-aba6-6c19f793cfa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:03.067083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310832c-d28a-47a2-aba6-6c19f793cfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:03.067352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310832c-d28a-47a2-aba6-6c19f793cfa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:03.068461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310832c-d28a-47a2-aba6-6c19f793cfa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:03.070059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310832c-d28a-47a2-aba6-6c19f793cfa4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:03.081113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1153/2324] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:03.086164 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6f661-6004-44df-82a3-320fa27a3701 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:03.101513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6f661-6004-44df-82a3-320fa27a3701 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:03.107708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc7ba2-39af-4b45-be38-5b76caf4c32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:03.108600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc7ba2-39af-4b45-be38-5b76caf4c32b admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:03.108884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc7ba2-39af-4b45-be38-5b76caf4c32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:03.109191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc7ba2-39af-4b45-be38-5b76caf4c32b admin admin] PATH_INFO: `/v3/projects/a07593d328db4b979a1671e1b3b6eb50/users/f9017ad76cf7475885315f3d514b7159/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:03.126710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc7ba2-39af-4b45-be38-5b76caf4c32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07593d328db4b979a1671e1b3b6eb50, user_id=f9017ad76cf7475885315f3d514b7159, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:03.130891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc7ba2-39af-4b45-be38-5b76caf4c32b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:03.148966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1172/2325] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:03 2022] PUT /identity/v3/projects/a07593d328db4b979a1671e1b3b6eb50/users/f9017ad76cf7475885315f3d514b7159/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:03.153072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310832c-d28a-47a2-aba6-6c19f793cfa4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:03.163293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310832c-d28a-47a2-aba6-6c19f793cfa4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:03.221743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb488a-7cad-4273-8d33-40043edf804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:03.223060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb488a-7cad-4273-8d33-40043edf804e admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:03.223060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb488a-7cad-4273-8d33-40043edf804e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:03.223397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb488a-7cad-4273-8d33-40043edf804e admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:03.224060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb488a-7cad-4273-8d33-40043edf804e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:03.225682 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb488a-7cad-4273-8d33-40043edf804e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:03.236663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1154/2326] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:03.243109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc7ba2-39af-4b45-be38-5b76caf4c32b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:03.257604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc7ba2-39af-4b45-be38-5b76caf4c32b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:03.264625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d52e1-036d-4042-ac5e-2f48a332cdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:03.265525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d52e1-036d-4042-ac5e-2f48a332cdec admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:03.265810 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d52e1-036d-4042-ac5e-2f48a332cdec admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:03.266082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d52e1-036d-4042-ac5e-2f48a332cdec admin admin] PATH_INFO: `/v3/projects/a07593d328db4b979a1671e1b3b6eb50/users/f9017ad76cf7475885315f3d514b7159/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:03.268387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d52e1-036d-4042-ac5e-2f48a332cdec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07593d328db4b979a1671e1b3b6eb50, user_id=f9017ad76cf7475885315f3d514b7159, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:03.271117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d52e1-036d-4042-ac5e-2f48a332cdec admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:03.282739 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1173/2327] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:03 2022] PUT /identity/v3/projects/a07593d328db4b979a1671e1b3b6eb50/users/f9017ad76cf7475885315f3d514b7159/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:03.294567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aca20e-c29e-42bb-9b14-36ba95abca84 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:03.294856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aca20e-c29e-42bb-9b14-36ba95abca84 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:03.295133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aca20e-c29e-42bb-9b14-36ba95abca84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:03.323780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d6aca20e-c29e-42bb-9b14-36ba95abca84 None None] MFA Rules not processed for user `f9017ad76cf7475885315f3d514b7159`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:03.325007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d6aca20e-c29e-42bb-9b14-36ba95abca84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:03.394809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1155/2328] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:03.407593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d52e1-036d-4042-ac5e-2f48a332cdec admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:03.409388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ba8d52e1-036d-4042-ac5e-2f48a332cdec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:03.475511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d52e1-036d-4042-ac5e-2f48a332cdec admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:03.481271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9470d-db1e-49c2-939d-3a95c89e3c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:03.481930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9470d-db1e-49c2-939d-3a95c89e3c46 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:03.482270 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9470d-db1e-49c2-939d-3a95c89e3c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:03.482744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9470d-db1e-49c2-939d-3a95c89e3c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:03.493324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9470d-db1e-49c2-939d-3a95c89e3c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:03.494900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9470d-db1e-49c2-939d-3a95c89e3c46 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:03.505636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1174/2329] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:03.854957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aca20e-c29e-42bb-9b14-36ba95abca84 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:03.864487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aca20e-c29e-42bb-9b14-36ba95abca84 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:03.869760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48bb38-d1ef-4652-82ef-4a88daf999ba tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9017ad76cf7475885315f3d514b7159', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a07593d328db4b979a1671e1b3b6eb50', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:03.870493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48bb38-d1ef-4652-82ef-4a88daf999ba tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:03.870780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48bb38-d1ef-4652-82ef-4a88daf999ba tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:03.871059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48bb38-d1ef-4652-82ef-4a88daf999ba tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:03.871797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48bb38-d1ef-4652-82ef-4a88daf999ba tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:07:03.872116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48bb38-d1ef-4652-82ef-4a88daf999ba tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:03.873735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48bb38-d1ef-4652-82ef-4a88daf999ba tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:03.887008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1156/2330] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:07:03.891338 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9470d-db1e-49c2-939d-3a95c89e3c46 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:03.900970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9470d-db1e-49c2-939d-3a95c89e3c46 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:03.912303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef32997-f02d-4bc6-9ed8-4538427dce45 tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9017ad76cf7475885315f3d514b7159', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a07593d328db4b979a1671e1b3b6eb50', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:03.913121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef32997-f02d-4bc6-9ed8-4538427dce45 tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:03.913487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef32997-f02d-4bc6-9ed8-4538427dce45 tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:03.913838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef32997-f02d-4bc6-9ed8-4538427dce45 tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:03.914677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef32997-f02d-4bc6-9ed8-4538427dce45 tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:03.916667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef32997-f02d-4bc6-9ed8-4538427dce45 tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:03.943558 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1175/2331] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:03 2022] POST /identity/v3/projects => generated 354 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:03.947749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48bb38-d1ef-4652-82ef-4a88daf999ba tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:03.958057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48bb38-d1ef-4652-82ef-4a88daf999ba tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:03.962010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:04.030273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cdee6-16c6-4147-890f-1a9df039c70f tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9017ad76cf7475885315f3d514b7159', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a07593d328db4b979a1671e1b3b6eb50', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:04.030946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cdee6-16c6-4147-890f-1a9df039c70f tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:04.031245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cdee6-16c6-4147-890f-1a9df039c70f tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:04.031529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cdee6-16c6-4147-890f-1a9df039c70f tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:04.032265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cdee6-16c6-4147-890f-1a9df039c70f tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:07:04.032632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cdee6-16c6-4147-890f-1a9df039c70f tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:04.034296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cdee6-16c6-4147-890f-1a9df039c70f tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:04.046783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1157/2332] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:07:04.082954 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef32997-f02d-4bc6-9ed8-4538427dce45 tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:04.093463 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef32997-f02d-4bc6-9ed8-4538427dce45 tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:04.099016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9017ad76cf7475885315f3d514b7159', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a07593d328db4b979a1671e1b3b6eb50', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:04.099987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:04.100279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:04.100596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] PATH_INFO: `/v3/projects/bb15851e7e47455da7d8833493da7622` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:04.101856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bb15851e7e47455da7d8833493da7622)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:04.103900 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:04.154072 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb15851e7e47455da7d8833493da7622', 'request_id': 'req-43d07901-d9e0-49a8-8a01-0230c0d0b79d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:04.162582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb15851e7e47455da7d8833493da7622', 'request_id': 'req-43d07901-d9e0-49a8-8a01-0230c0d0b79d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:04.168303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb15851e7e47455da7d8833493da7622', 'request_id': 'req-43d07901-d9e0-49a8-8a01-0230c0d0b79d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:04.176444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:04.180462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] The token cache is being invalidate because project bb15851e7e47455da7d8833493da7622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:04.180752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:04.181852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43d07901-d9e0-49a8-8a01-0230c0d0b79d tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:04.183887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1176/2333] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:04 2022] DELETE /identity/v3/projects/bb15851e7e47455da7d8833493da7622 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:04.385888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cdee6-16c6-4147-890f-1a9df039c70f tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:04.387952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-336cdee6-16c6-4147-890f-1a9df039c70f tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:04.455643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cdee6-16c6-4147-890f-1a9df039c70f tempest-VolumeQuotaClassesTest-1158510048 tempest-VolumeQuotaClassesTest-1158510048-project-admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:04.512784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:04.513665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:04.513946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:04.514257 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] PATH_INFO: `/v3/users/900f536cc6a0423294ed37089c8e326a` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:04.515503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] RBAC: Authorizing `identity:delete_user(user_id=900f536cc6a0423294ed37089c8e326a)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:04.517463 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:04.577467 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '900f536cc6a0423294ed37089c8e326a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:04.578198 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '900f536cc6a0423294ed37089c8e326a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:04.580944 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:04.589249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '900f536cc6a0423294ed37089c8e326a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:04.590273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '900f536cc6a0423294ed37089c8e326a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:04.599311 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '900f536cc6a0423294ed37089c8e326a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:04.619732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1158/2334] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:04 2022] DELETE /identity/v3/users/900f536cc6a0423294ed37089c8e326a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:04.624997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2f32f-11da-4624-8f7f-7d62e5f3b33f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:04.625285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2f32f-11da-4624-8f7f-7d62e5f3b33f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:04.625564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2f32f-11da-4624-8f7f-7d62e5f3b33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:04.644159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-16f2f32f-11da-4624-8f7f-7d62e5f3b33f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:04.645366 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-16f2f32f-11da-4624-8f7f-7d62e5f3b33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:04.705578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1177/2335] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:04.716575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:04.718412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:04.775719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98883fd-0a2e-4bea-acdc-592ec821041d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:04.780004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c7707-0d11-4361-b0b8-51f209c66954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:04.780659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c7707-0d11-4361-b0b8-51f209c66954 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:04.780934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c7707-0d11-4361-b0b8-51f209c66954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:04.781220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c7707-0d11-4361-b0b8-51f209c66954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:04.791844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c7707-0d11-4361-b0b8-51f209c66954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:04.793332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c7707-0d11-4361-b0b8-51f209c66954 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:04.803966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1159/2336] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:05.117803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2f32f-11da-4624-8f7f-7d62e5f3b33f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:05.123959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-16f2f32f-11da-4624-8f7f-7d62e5f3b33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:05.164201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2f32f-11da-4624-8f7f-7d62e5f3b33f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:05.171587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:05.172621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:05.173089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:05.173568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] PATH_INFO: `/v3/projects/c75ea30ba8e14ba1aa4f3ca36d38fd39` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:05.176946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75ea30ba8e14ba1aa4f3ca36d38fd39)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:05.178897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:05.241507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75ea30ba8e14ba1aa4f3ca36d38fd39', 'request_id': 'req-0e21e1f7-458c-47f2-9abf-15e35759c5db'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.251153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75ea30ba8e14ba1aa4f3ca36d38fd39', 'request_id': 'req-0e21e1f7-458c-47f2-9abf-15e35759c5db'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.257826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75ea30ba8e14ba1aa4f3ca36d38fd39', 'request_id': 'req-0e21e1f7-458c-47f2-9abf-15e35759c5db'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.273302 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] The token cache is being invalidate because project c75ea30ba8e14ba1aa4f3ca36d38fd39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:05.273592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.274298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.275924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1178/2337] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:05 2022] DELETE /identity/v3/projects/c75ea30ba8e14ba1aa4f3ca36d38fd39 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:05.280389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c7707-0d11-4361-b0b8-51f209c66954 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:05.290251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c7707-0d11-4361-b0b8-51f209c66954 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:05.356993 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:05.357802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:05.358105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:05.358393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] PATH_INFO: `/v3/users/f9017ad76cf7475885315f3d514b7159` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:05.359827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9017ad76cf7475885315f3d514b7159)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:05.361800 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:05.413403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9017ad76cf7475885315f3d514b7159'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.414054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9017ad76cf7475885315f3d514b7159'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.422334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9017ad76cf7475885315f3d514b7159'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.422973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9017ad76cf7475885315f3d514b7159'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.429934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ce376100-6d0d-4c84-843e-59bfa696c217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9017ad76cf7475885315f3d514b7159'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.437372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:05.449866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1160/2338] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:05 2022] DELETE /identity/v3/users/f9017ad76cf7475885315f3d514b7159 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:05.732257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:05.733939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:05.750541 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21e1f7-458c-47f2-9abf-15e35759c5db admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:05.807352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:05.808341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:05.808624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:05.808899 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] PATH_INFO: `/v3/projects/a07593d328db4b979a1671e1b3b6eb50` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:05.810107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07593d328db4b979a1671e1b3b6eb50)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:05.812048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:05.845591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:05.863647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07593d328db4b979a1671e1b3b6eb50', 'request_id': 'req-201c9c19-7e55-40ad-be25-233a28dad9f9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.872082 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07593d328db4b979a1671e1b3b6eb50', 'request_id': 'req-201c9c19-7e55-40ad-be25-233a28dad9f9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.879452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07593d328db4b979a1671e1b3b6eb50', 'request_id': 'req-201c9c19-7e55-40ad-be25-233a28dad9f9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.891320 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] The token cache is being invalidate because project a07593d328db4b979a1671e1b3b6eb50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:05.891608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.892265 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-201c9c19-7e55-40ad-be25-233a28dad9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:05.894069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1179/2339] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:05 2022] DELETE /identity/v3/projects/a07593d328db4b979a1671e1b3b6eb50 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:05.898893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed641065-0821-49f3-85ba-4fdce9aa0f5e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:05.899180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed641065-0821-49f3-85ba-4fdce9aa0f5e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:05.899457 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed641065-0821-49f3-85ba-4fdce9aa0f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:05.923820 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ed641065-0821-49f3-85ba-4fdce9aa0f5e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:05.925058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ed641065-0821-49f3-85ba-4fdce9aa0f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:05.986695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1161/2340] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:05.994545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6ee4c-c3b1-47b0-bc9f-4b4a9ff44a55 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:05.994903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6ee4c-c3b1-47b0-bc9f-4b4a9ff44a55 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:05.995185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6ee4c-c3b1-47b0-bc9f-4b4a9ff44a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.014935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f6f6ee4c-c3b1-47b0-bc9f-4b4a9ff44a55 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:06.016129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f6f6ee4c-c3b1-47b0-bc9f-4b4a9ff44a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:06.025041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1180/2341] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:06.031234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed641065-0821-49f3-85ba-4fdce9aa0f5e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.041604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed641065-0821-49f3-85ba-4fdce9aa0f5e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.047731 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45810969-3d68-41bd-8c97-4ada8d7d8acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.048404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810969-3d68-41bd-8c97-4ada8d7d8acd admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.048663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810969-3d68-41bd-8c97-4ada8d7d8acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.048959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810969-3d68-41bd-8c97-4ada8d7d8acd admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.049666 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45810969-3d68-41bd-8c97-4ada8d7d8acd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:07:06.049975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45810969-3d68-41bd-8c97-4ada8d7d8acd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.051640 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45810969-3d68-41bd-8c97-4ada8d7d8acd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.064508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1162/2342] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:07:06.069244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6ee4c-c3b1-47b0-bc9f-4b4a9ff44a55 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.079167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6ee4c-c3b1-47b0-bc9f-4b4a9ff44a55 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.084279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bd510-8540-471c-aca0-4ccff1b0d074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.084925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bd510-8540-471c-aca0-4ccff1b0d074 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.085199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bd510-8540-471c-aca0-4ccff1b0d074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.085468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bd510-8540-471c-aca0-4ccff1b0d074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.086200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6bd510-8540-471c-aca0-4ccff1b0d074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.088105 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6bd510-8540-471c-aca0-4ccff1b0d074 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.118874 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1181/2343] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:06 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:06.123559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45810969-3d68-41bd-8c97-4ada8d7d8acd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.133730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45810969-3d68-41bd-8c97-4ada8d7d8acd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.192770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bc811-bc8a-494b-956b-49e16eb5d9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.193793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bc811-bc8a-494b-956b-49e16eb5d9aa admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.194077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bc811-bc8a-494b-956b-49e16eb5d9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.194389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bc811-bc8a-494b-956b-49e16eb5d9aa admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.195090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bc811-bc8a-494b-956b-49e16eb5d9aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.197021 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bc811-bc8a-494b-956b-49e16eb5d9aa admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.222052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1163/2344] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:06 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:06.223341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bd510-8540-471c-aca0-4ccff1b0d074 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.234261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bd510-8540-471c-aca0-4ccff1b0d074 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.239334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb8730-ec0b-4848-ac7e-c444e7cea845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.240010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb8730-ec0b-4848-ac7e-c444e7cea845 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.240298 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb8730-ec0b-4848-ac7e-c444e7cea845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.240581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb8730-ec0b-4848-ac7e-c444e7cea845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.241318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb8730-ec0b-4848-ac7e-c444e7cea845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.243027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb8730-ec0b-4848-ac7e-c444e7cea845 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.257281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1182/2345] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:06.258566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bc811-bc8a-494b-956b-49e16eb5d9aa admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.270630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bc811-bc8a-494b-956b-49e16eb5d9aa admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.275478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fff6f-c814-4efa-ab2d-cbacfafec807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.276275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fff6f-c814-4efa-ab2d-cbacfafec807 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.277135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fff6f-c814-4efa-ab2d-cbacfafec807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.277525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fff6f-c814-4efa-ab2d-cbacfafec807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.278289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fff6f-c814-4efa-ab2d-cbacfafec807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.279928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fff6f-c814-4efa-ab2d-cbacfafec807 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.290299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1164/2346] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:06.295070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb8730-ec0b-4848-ac7e-c444e7cea845 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.305533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb8730-ec0b-4848-ac7e-c444e7cea845 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.310859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5d237-1fd6-4882-afab-fe0724981860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.311746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5d237-1fd6-4882-afab-fe0724981860 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.312077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5d237-1fd6-4882-afab-fe0724981860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.312350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5d237-1fd6-4882-afab-fe0724981860 admin admin] PATH_INFO: `/v3/projects/42a3cfe5aa2440a492ce3e6585bb969c/users/7f3a611a84684523bbcfbde62d375ad9/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.328690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5d237-1fd6-4882-afab-fe0724981860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a3cfe5aa2440a492ce3e6585bb969c, user_id=7f3a611a84684523bbcfbde62d375ad9, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.331384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5d237-1fd6-4882-afab-fe0724981860 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.346414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1183/2347] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:06 2022] PUT /identity/v3/projects/42a3cfe5aa2440a492ce3e6585bb969c/users/7f3a611a84684523bbcfbde62d375ad9/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:07:06.357654 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1e301-e634-4d23-95a5-c81f233e8f84 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.357945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1e301-e634-4d23-95a5-c81f233e8f84 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.358261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1e301-e634-4d23-95a5-c81f233e8f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.387780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-67b1e301-e634-4d23-95a5-c81f233e8f84 None None] MFA Rules not processed for user `7f3a611a84684523bbcfbde62d375ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:06.388925 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-67b1e301-e634-4d23-95a5-c81f233e8f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:06.438801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:06.461727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1165/2348] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:06.466490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5d237-1fd6-4882-afab-fe0724981860 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.539950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5d237-1fd6-4882-afab-fe0724981860 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.545325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39254049-f7c0-4ec3-ac93-314ba5082a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.545992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39254049-f7c0-4ec3-ac93-314ba5082a9b admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.546308 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39254049-f7c0-4ec3-ac93-314ba5082a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.546586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39254049-f7c0-4ec3-ac93-314ba5082a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.547271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39254049-f7c0-4ec3-ac93-314ba5082a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.549192 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39254049-f7c0-4ec3-ac93-314ba5082a9b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.587120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1184/2349] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:06 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:06.592438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1e301-e634-4d23-95a5-c81f233e8f84 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.603766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1e301-e634-4d23-95a5-c81f233e8f84 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.665544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad4db9-4ffd-4396-be0b-5756853d0f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.666267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad4db9-4ffd-4396-be0b-5756853d0f1a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.666549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad4db9-4ffd-4396-be0b-5756853d0f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.667031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad4db9-4ffd-4396-be0b-5756853d0f1a admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.667835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad4db9-4ffd-4396-be0b-5756853d0f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.669832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad4db9-4ffd-4396-be0b-5756853d0f1a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.690707 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1166/2350] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:06 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:06.694949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39254049-f7c0-4ec3-ac93-314ba5082a9b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.706648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39254049-f7c0-4ec3-ac93-314ba5082a9b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.711898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a79df-fdf2-486a-9037-5bc50169d600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.712653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a79df-fdf2-486a-9037-5bc50169d600 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.712974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a79df-fdf2-486a-9037-5bc50169d600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.713340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a79df-fdf2-486a-9037-5bc50169d600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.714476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a79df-fdf2-486a-9037-5bc50169d600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.716410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a79df-fdf2-486a-9037-5bc50169d600 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.728220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1185/2351] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:06.732646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad4db9-4ffd-4396-be0b-5756853d0f1a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.743051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad4db9-4ffd-4396-be0b-5756853d0f1a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.750404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e2e94-da6d-4be9-aa8f-b1723ca1dc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.751071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e2e94-da6d-4be9-aa8f-b1723ca1dc31 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.751343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e2e94-da6d-4be9-aa8f-b1723ca1dc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.751616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e2e94-da6d-4be9-aa8f-b1723ca1dc31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.752344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e2e94-da6d-4be9-aa8f-b1723ca1dc31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.753966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e2e94-da6d-4be9-aa8f-b1723ca1dc31 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.764592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1167/2352] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:06.769919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a79df-fdf2-486a-9037-5bc50169d600 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:06.783217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a79df-fdf2-486a-9037-5bc50169d600 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:06.788757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15b9cb-20d9-4b05-b810-a353687b5420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:06.789676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15b9cb-20d9-4b05-b810-a353687b5420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.789961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15b9cb-20d9-4b05-b810-a353687b5420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.790280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15b9cb-20d9-4b05-b810-a353687b5420 admin admin] PATH_INFO: `/v3/projects/13009379652744e3a5761afefb24a580/users/934fed89eae24d90bcd90f8fdb855ecf/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.806607 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15b9cb-20d9-4b05-b810-a353687b5420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13009379652744e3a5761afefb24a580, user_id=934fed89eae24d90bcd90f8fdb855ecf, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:06.809455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15b9cb-20d9-4b05-b810-a353687b5420 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:06.820794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:06.826343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1186/2353] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:06 2022] PUT /identity/v3/projects/13009379652744e3a5761afefb24a580/users/934fed89eae24d90bcd90f8fdb855ecf/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:06.837601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a16aa2-09bd-42ce-b24e-5776e8dd07ce None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:06.837917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a16aa2-09bd-42ce-b24e-5776e8dd07ce None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:06.838242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a16aa2-09bd-42ce-b24e-5776e8dd07ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:06.872181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-15a16aa2-09bd-42ce-b24e-5776e8dd07ce None None] MFA Rules not processed for user `934fed89eae24d90bcd90f8fdb855ecf`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:06.873840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-15a16aa2-09bd-42ce-b24e-5776e8dd07ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:06.949406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1168/2354] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:06.954043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15b9cb-20d9-4b05-b810-a353687b5420 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:07.029167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15b9cb-20d9-4b05-b810-a353687b5420 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:07.034118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f775a78-380f-4536-8183-d983eabcbd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:07.034848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f775a78-380f-4536-8183-d983eabcbd73 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:07.035136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f775a78-380f-4536-8183-d983eabcbd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:07.035412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f775a78-380f-4536-8183-d983eabcbd73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:07.036161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f775a78-380f-4536-8183-d983eabcbd73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:07.038129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f775a78-380f-4536-8183-d983eabcbd73 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:07.066449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1187/2355] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:06 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:07:07.071037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a16aa2-09bd-42ce-b24e-5776e8dd07ce None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:07.082964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a16aa2-09bd-42ce-b24e-5776e8dd07ce None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:07.135706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:07.147579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332385da-7fb1-4922-9140-481abb2406de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:07.148266 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332385da-7fb1-4922-9140-481abb2406de admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:07.148542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332385da-7fb1-4922-9140-481abb2406de admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:07.148834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332385da-7fb1-4922-9140-481abb2406de admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:07.149562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332385da-7fb1-4922-9140-481abb2406de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:07.152610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332385da-7fb1-4922-9140-481abb2406de admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:07.179000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1169/2356] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:07 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:07.182998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f775a78-380f-4536-8183-d983eabcbd73 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:07.194347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f775a78-380f-4536-8183-d983eabcbd73 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:07.200360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f63fa-b229-4bb3-b087-258566956229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:07.201058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f63fa-b229-4bb3-b087-258566956229 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:07.201322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f63fa-b229-4bb3-b087-258566956229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:07.201574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f63fa-b229-4bb3-b087-258566956229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:07.202349 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f63fa-b229-4bb3-b087-258566956229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:07.203982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f63fa-b229-4bb3-b087-258566956229 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:07.215572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1188/2357] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:07.219605 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332385da-7fb1-4922-9140-481abb2406de admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:07.230288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332385da-7fb1-4922-9140-481abb2406de admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:07.234887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be8da2-fc76-4250-ac71-211760540f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:07.235779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be8da2-fc76-4250-ac71-211760540f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:07.236057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be8da2-fc76-4250-ac71-211760540f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:07.236370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be8da2-fc76-4250-ac71-211760540f71 admin admin] PATH_INFO: `/v3/projects/49d9d7af744b4558a0beac722bab3ffe/users/0bf7c8bfc6b54015b03ebbeb3406e7a4/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:07.252302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be8da2-fc76-4250-ac71-211760540f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d9d7af744b4558a0beac722bab3ffe, user_id=0bf7c8bfc6b54015b03ebbeb3406e7a4, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:07.254644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be8da2-fc76-4250-ac71-211760540f71 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:07.268700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1170/2358] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:07 2022] PUT /identity/v3/projects/49d9d7af744b4558a0beac722bab3ffe/users/0bf7c8bfc6b54015b03ebbeb3406e7a4/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:07.272925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f63fa-b229-4bb3-b087-258566956229 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:07.283326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f63fa-b229-4bb3-b087-258566956229 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:07.341745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272f542-1361-4eef-890f-b9086110444f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:07.342544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272f542-1361-4eef-890f-b9086110444f admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:07.342827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272f542-1361-4eef-890f-b9086110444f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:07.343104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272f542-1361-4eef-890f-b9086110444f admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:07.343860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272f542-1361-4eef-890f-b9086110444f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:07.345484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272f542-1361-4eef-890f-b9086110444f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:07.355580 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1189/2359] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:07.360101 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be8da2-fc76-4250-ac71-211760540f71 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:07.371427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be8da2-fc76-4250-ac71-211760540f71 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:07.377285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab104114-e19b-4d26-9f12-eb217ffc0252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:07.378120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab104114-e19b-4d26-9f12-eb217ffc0252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:07.378419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab104114-e19b-4d26-9f12-eb217ffc0252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:07.378696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab104114-e19b-4d26-9f12-eb217ffc0252 admin admin] PATH_INFO: `/v3/projects/49d9d7af744b4558a0beac722bab3ffe/users/0bf7c8bfc6b54015b03ebbeb3406e7a4/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:07.380933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab104114-e19b-4d26-9f12-eb217ffc0252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d9d7af744b4558a0beac722bab3ffe, user_id=0bf7c8bfc6b54015b03ebbeb3406e7a4, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:07.383640 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab104114-e19b-4d26-9f12-eb217ffc0252 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:07.395592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1171/2360] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:07 2022] PUT /identity/v3/projects/49d9d7af744b4558a0beac722bab3ffe/users/0bf7c8bfc6b54015b03ebbeb3406e7a4/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:07.406639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83f7df-ecdc-48e8-9468-b6db66818298 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:07.406942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83f7df-ecdc-48e8-9468-b6db66818298 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:07.407199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83f7df-ecdc-48e8-9468-b6db66818298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:07.434806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4e83f7df-ecdc-48e8-9468-b6db66818298 None None] MFA Rules not processed for user `0bf7c8bfc6b54015b03ebbeb3406e7a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:07.435949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4e83f7df-ecdc-48e8-9468-b6db66818298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:07.500352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:07.511374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1190/2361] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:07.523229 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab104114-e19b-4d26-9f12-eb217ffc0252 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:07.525775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ab104114-e19b-4d26-9f12-eb217ffc0252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:07.586473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab104114-e19b-4d26-9f12-eb217ffc0252 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:07.590385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae4f54-0a35-4b5f-90ad-6b2e0f867bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:07.591162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae4f54-0a35-4b5f-90ad-6b2e0f867bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:07.591558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae4f54-0a35-4b5f-90ad-6b2e0f867bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:07.591844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae4f54-0a35-4b5f-90ad-6b2e0f867bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:07.603502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae4f54-0a35-4b5f-90ad-6b2e0f867bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:07.605498 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae4f54-0a35-4b5f-90ad-6b2e0f867bf8 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:07.616038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1172/2362] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:08.155213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83f7df-ecdc-48e8-9468-b6db66818298 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:08.165143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83f7df-ecdc-48e8-9468-b6db66818298 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:08.169493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea6178-2092-4e9c-85c9-3b812f08e1f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:08.170128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea6178-2092-4e9c-85c9-3b812f08e1f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:08.170430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea6178-2092-4e9c-85c9-3b812f08e1f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:08.170675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea6178-2092-4e9c-85c9-3b812f08e1f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:08.231135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea6178-2092-4e9c-85c9-3b812f08e1f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:08.232502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea6178-2092-4e9c-85c9-3b812f08e1f4 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:08.242576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1191/2363] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:17.611271 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=46,keystone:INSERT=5 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:18.238121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=30 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:19.972903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41aebb-c5f9-4e05-abd3-705c4b597bbd None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:19.973452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41aebb-c5f9-4e05-abd3-705c4b597bbd None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:19.973549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41aebb-c5f9-4e05-abd3-705c4b597bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:19.992610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3a41aebb-c5f9-4e05-abd3-705c4b597bbd None None] MFA Rules not processed for user `ee21ef9eacd24177a852f28d17376191`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:19.993830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3a41aebb-c5f9-4e05-abd3-705c4b597bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:20.049839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1173/2364] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:23.002370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea6178-2092-4e9c-85c9-3b812f08e1f4 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:23.011950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea6178-2092-4e9c-85c9-3b812f08e1f4 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:23.018069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481e114-b829-4f56-9242-6af84a03a4ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:23.018760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481e114-b829-4f56-9242-6af84a03a4ff service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:23.019244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481e114-b829-4f56-9242-6af84a03a4ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:23.019936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481e114-b829-4f56-9242-6af84a03a4ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:23.094718 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481e114-b829-4f56-9242-6af84a03a4ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:23.097657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481e114-b829-4f56-9242-6af84a03a4ff service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:23.113770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1192/2365] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:23.957208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7f325-bb02-472e-9eb4-4af972a472c8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:23.958190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7f325-bb02-472e-9eb4-4af972a472c8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:23.958747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7f325-bb02-472e-9eb4-4af972a472c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:23.981446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-81c7f325-bb02-472e-9eb4-4af972a472c8 None None] MFA Rules not processed for user `0c4a4ce86beb47c2b7910264a9238c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:23.983387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-81c7f325-bb02-472e-9eb4-4af972a472c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:24.064276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1174/2366] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:25.252311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481e114-b829-4f56-9242-6af84a03a4ff service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:25.254109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b481e114-b829-4f56-9242-6af84a03a4ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:25.415821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481e114-b829-4f56-9242-6af84a03a4ff service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:25.426826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:25.428118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:25.428716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:25.429132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] PATH_INFO: `/v3/users/9c71707487fa4685994d1677ba69b127` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:25.430681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c71707487fa4685994d1677ba69b127)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:25.432495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:25.510624 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c71707487fa4685994d1677ba69b127'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:25.519427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c71707487fa4685994d1677ba69b127'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:25.520264 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c71707487fa4685994d1677ba69b127'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:25.521199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c71707487fa4685994d1677ba69b127'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:25.538836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c71707487fa4685994d1677ba69b127'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:25.556685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1193/2367] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:25 2022] DELETE /identity/v3/users/9c71707487fa4685994d1677ba69b127 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:25.561865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680aee2-4dc9-42f1-9232-60b963538431 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:25.562762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680aee2-4dc9-42f1-9232-60b963538431 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:25.563040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680aee2-4dc9-42f1-9232-60b963538431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:25.587854 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b680aee2-4dc9-42f1-9232-60b963538431 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:25.589176 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b680aee2-4dc9-42f1-9232-60b963538431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:25.681085 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1175/2368] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:25.692042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:25.694978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:25.728618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:25.776944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5087b66-6447-4523-bc04-94ff23cbeb16 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:25.782168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4364551-624c-486e-a463-cc09c70306c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:25.782888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364551-624c-486e-a463-cc09c70306c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:25.783246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364551-624c-486e-a463-cc09c70306c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:25.783592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4364551-624c-486e-a463-cc09c70306c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:25.795851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4364551-624c-486e-a463-cc09c70306c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:25.797902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4364551-624c-486e-a463-cc09c70306c9 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:25.808037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1194/2369] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:25.869309 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680aee2-4dc9-42f1-9232-60b963538431 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:25.870753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b680aee2-4dc9-42f1-9232-60b963538431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:25.883470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680aee2-4dc9-42f1-9232-60b963538431 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:25.888151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:25.889631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:25.889914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:25.890228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] PATH_INFO: `/v3/users/7f3a611a84684523bbcfbde62d375ad9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:25.891444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f3a611a84684523bbcfbde62d375ad9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:25.893408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:25.948837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:25.950544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f3a611a84684523bbcfbde62d375ad9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:25.951209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f3a611a84684523bbcfbde62d375ad9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:25.967088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f3a611a84684523bbcfbde62d375ad9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:25.967701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f3a611a84684523bbcfbde62d375ad9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:25.976536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f3a611a84684523bbcfbde62d375ad9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:25.998548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1176/2370] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:25 2022] DELETE /identity/v3/users/7f3a611a84684523bbcfbde62d375ad9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:26.003217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b478379-56f4-465c-befb-38994f7b7180 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.003507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b478379-56f4-465c-befb-38994f7b7180 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.003781 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b478379-56f4-465c-befb-38994f7b7180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.021820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5b478379-56f4-465c-befb-38994f7b7180 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:26.023131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5b478379-56f4-465c-befb-38994f7b7180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:26.088655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1195/2371] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:26.095765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.097200 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:26.100625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b478379-56f4-465c-befb-38994f7b7180 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.102257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5b478379-56f4-465c-befb-38994f7b7180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:26.115850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fd9f1-1db2-4887-bbb9-0d4113ad6bfe admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.121769 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.122701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.122980 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.123402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] PATH_INFO: `/v3/projects/a821e7003d274be79ee94c43df9d17c8` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.125281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a821e7003d274be79ee94c43df9d17c8)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.127245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.173949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b478379-56f4-465c-befb-38994f7b7180 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.178891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a821e7003d274be79ee94c43df9d17c8', 'request_id': 'req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.187720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a821e7003d274be79ee94c43df9d17c8', 'request_id': 'req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.194302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a821e7003d274be79ee94c43df9d17c8', 'request_id': 'req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.206165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] The token cache is being invalidate because project a821e7003d274be79ee94c43df9d17c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:26.206571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.207280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-dd4710fc-138b-423b-b380-d9e3f0dd7ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.208997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1177/2372] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:26 2022] DELETE /identity/v3/projects/a821e7003d274be79ee94c43df9d17c8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:26.214011 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd05647-374c-41f6-a893-4e156a61fad6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.214324 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd05647-374c-41f6-a893-4e156a61fad6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.214598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd05647-374c-41f6-a893-4e156a61fad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.227305 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924924fb-8ed6-4be2-9d05-48e5038b16bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.227980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924924fb-8ed6-4be2-9d05-48e5038b16bf service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.228260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924924fb-8ed6-4be2-9d05-48e5038b16bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.228533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924924fb-8ed6-4be2-9d05-48e5038b16bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.230302 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-924924fb-8ed6-4be2-9d05-48e5038b16bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:26.242539 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-bcd05647-374c-41f6-a893-4e156a61fad6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:26.243719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-bcd05647-374c-41f6-a893-4e156a61fad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:26.262117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:26.296979 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924924fb-8ed6-4be2-9d05-48e5038b16bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.298519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924924fb-8ed6-4be2-9d05-48e5038b16bf service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.304551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1178/2373] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:26.308305 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1196/2374] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:07:26.312344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5061b9b-cf8e-4aa9-b229-7053b1a833b4 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.312630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5061b9b-cf8e-4aa9-b229-7053b1a833b4 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.312913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5061b9b-cf8e-4aa9-b229-7053b1a833b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.331939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e5061b9b-cf8e-4aa9-b229-7053b1a833b4 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:26.333131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e5061b9b-cf8e-4aa9-b229-7053b1a833b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:26.341341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1179/2375] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:26.346211 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924924fb-8ed6-4be2-9d05-48e5038b16bf service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.356402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924924fb-8ed6-4be2-9d05-48e5038b16bf service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.361690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b812ba-348a-43e2-801b-e8de5f07b11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.362392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b812ba-348a-43e2-801b-e8de5f07b11a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.362669 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b812ba-348a-43e2-801b-e8de5f07b11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.362941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b812ba-348a-43e2-801b-e8de5f07b11a admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.363701 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b812ba-348a-43e2-801b-e8de5f07b11a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:07:26.364002 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b812ba-348a-43e2-801b-e8de5f07b11a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.365554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b812ba-348a-43e2-801b-e8de5f07b11a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.378502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1197/2376] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:07:26.382638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5061b9b-cf8e-4aa9-b229-7053b1a833b4 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.392245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5061b9b-cf8e-4aa9-b229-7053b1a833b4 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.397246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668734b-1837-4196-980e-ff2df8a11626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.397973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668734b-1837-4196-980e-ff2df8a11626 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.398352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668734b-1837-4196-980e-ff2df8a11626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.398654 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668734b-1837-4196-980e-ff2df8a11626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.399483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668734b-1837-4196-980e-ff2df8a11626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.401223 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668734b-1837-4196-980e-ff2df8a11626 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.429698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1180/2377] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:26 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:07:26.433921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b812ba-348a-43e2-801b-e8de5f07b11a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.447886 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b812ba-348a-43e2-801b-e8de5f07b11a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.546434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf1b37-9c49-40a0-b411-d44519630294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.547173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf1b37-9c49-40a0-b411-d44519630294 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.547455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf1b37-9c49-40a0-b411-d44519630294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.547732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf1b37-9c49-40a0-b411-d44519630294 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.548458 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf1b37-9c49-40a0-b411-d44519630294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.550521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf1b37-9c49-40a0-b411-d44519630294 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.572847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1198/2378] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:26 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:26.576618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668734b-1837-4196-980e-ff2df8a11626 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.588023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668734b-1837-4196-980e-ff2df8a11626 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.594756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49374df8-40d1-4284-9951-236f77fd883f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.595444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49374df8-40d1-4284-9951-236f77fd883f admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.595721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49374df8-40d1-4284-9951-236f77fd883f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.596000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49374df8-40d1-4284-9951-236f77fd883f admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.596884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49374df8-40d1-4284-9951-236f77fd883f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.599136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49374df8-40d1-4284-9951-236f77fd883f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.601843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:26.614251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf1b37-9c49-40a0-b411-d44519630294 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.616134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f9bf1b37-9c49-40a0-b411-d44519630294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:26.616469 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1181/2379] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:26.617406 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49374df8-40d1-4284-9951-236f77fd883f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.628055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49374df8-40d1-4284-9951-236f77fd883f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.632891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb3225-b3d2-4db5-927f-7ca491086afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.633567 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb3225-b3d2-4db5-927f-7ca491086afb admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.633846 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb3225-b3d2-4db5-927f-7ca491086afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.634192 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb3225-b3d2-4db5-927f-7ca491086afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.634936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb3225-b3d2-4db5-927f-7ca491086afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.635620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf1b37-9c49-40a0-b411-d44519630294 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.636536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb3225-b3d2-4db5-927f-7ca491086afb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.641110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.642347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.642549 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.642890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] PATH_INFO: `/v3/projects/42a3cfe5aa2440a492ce3e6585bb969c` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.645060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a3cfe5aa2440a492ce3e6585bb969c)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.647043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.650579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1182/2380] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:26.654721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb3225-b3d2-4db5-927f-7ca491086afb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.664352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb3225-b3d2-4db5-927f-7ca491086afb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.669211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8fffd-7f15-4450-a6ce-0e7a125758bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.670217 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8fffd-7f15-4450-a6ce-0e7a125758bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.670507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8fffd-7f15-4450-a6ce-0e7a125758bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.670790 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8fffd-7f15-4450-a6ce-0e7a125758bd admin admin] PATH_INFO: `/v3/projects/51c3f69c523b49e68d7d4d09f3236c4e/users/fdff9a743af948f88d60a6fd80795c06/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.685961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8fffd-7f15-4450-a6ce-0e7a125758bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c3f69c523b49e68d7d4d09f3236c4e, user_id=fdff9a743af948f88d60a6fd80795c06, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.688670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8fffd-7f15-4450-a6ce-0e7a125758bd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.697023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a3cfe5aa2440a492ce3e6585bb969c', 'request_id': 'req-fb610877-0124-44a2-9de4-2103445fe9f4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.703259 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1183/2381] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:26 2022] PUT /identity/v3/projects/51c3f69c523b49e68d7d4d09f3236c4e/users/fdff9a743af948f88d60a6fd80795c06/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:07:26.705381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a3cfe5aa2440a492ce3e6585bb969c', 'request_id': 'req-fb610877-0124-44a2-9de4-2103445fe9f4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.708234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81bf6d-2d3a-4dad-bc70-cf6a79418512 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.708528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81bf6d-2d3a-4dad-bc70-cf6a79418512 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.708805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81bf6d-2d3a-4dad-bc70-cf6a79418512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.711675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a3cfe5aa2440a492ce3e6585bb969c', 'request_id': 'req-fb610877-0124-44a2-9de4-2103445fe9f4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.726374 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] The token cache is being invalidate because project 42a3cfe5aa2440a492ce3e6585bb969c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:26.726687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.727344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.728903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1199/2382] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:26 2022] DELETE /identity/v3/projects/42a3cfe5aa2440a492ce3e6585bb969c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:07:26.729694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-9f81bf6d-2d3a-4dad-bc70-cf6a79418512 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:26.730932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9f81bf6d-2d3a-4dad-bc70-cf6a79418512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:26.733079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.734386 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:26.766789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:26.805436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1184/2383] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:26.813295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb610877-0124-44a2-9de4-2103445fe9f4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.814182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81bf6d-2d3a-4dad-bc70-cf6a79418512 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:26.815543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-9f81bf6d-2d3a-4dad-bc70-cf6a79418512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:26.819065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.819961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.820245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.820521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] PATH_INFO: `/v3/users/934fed89eae24d90bcd90f8fdb855ecf` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.821745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=934fed89eae24d90bcd90f8fdb855ecf)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.823881 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.877737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934fed89eae24d90bcd90f8fdb855ecf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.882139 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81bf6d-2d3a-4dad-bc70-cf6a79418512 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:26.886194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d136a4-2378-491a-9338-927d164dcbf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:26.886948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d136a4-2378-491a-9338-927d164dcbf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:26.887235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d136a4-2378-491a-9338-927d164dcbf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:26.887520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d136a4-2378-491a-9338-927d164dcbf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:26.889287 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934fed89eae24d90bcd90f8fdb855ecf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.889976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934fed89eae24d90bcd90f8fdb855ecf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.890644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934fed89eae24d90bcd90f8fdb855ecf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.898928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d136a4-2378-491a-9338-927d164dcbf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:26.900344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d136a4-2378-491a-9338-927d164dcbf0 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:26.907677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934fed89eae24d90bcd90f8fdb855ecf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:26.912864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1185/2384] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:07:26.929093 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1200/2385] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:26 2022] DELETE /identity/v3/users/934fed89eae24d90bcd90f8fdb855ecf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:27.224059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d136a4-2378-491a-9338-927d164dcbf0 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:27.235175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d136a4-2378-491a-9338-927d164dcbf0 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:27.310525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:27.311450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:27.311743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:27.312020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] PATH_INFO: `/v3/projects/13009379652744e3a5761afefb24a580` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:27.313207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] RBAC: Authorizing `identity:delete_project(project_id=13009379652744e3a5761afefb24a580)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:27.316230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:27.330708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:27.383133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13009379652744e3a5761afefb24a580', 'request_id': 'req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:27.393071 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13009379652744e3a5761afefb24a580', 'request_id': 'req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:27.401968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13009379652744e3a5761afefb24a580', 'request_id': 'req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:27.415655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] The token cache is being invalidate because project 13009379652744e3a5761afefb24a580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:27.416190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:27.416858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:27.419017 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1186/2386] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:27 2022] DELETE /identity/v3/projects/13009379652744e3a5761afefb24a580 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:27.423518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:27.425124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:27.439281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f24277-c903-4a67-82fa-5ab302ed5814 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:27.513910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:27.515642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:27.515967 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:27.516247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] PATH_INFO: `/v3/users/0bf7c8bfc6b54015b03ebbeb3406e7a4` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:27.517533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf7c8bfc6b54015b03ebbeb3406e7a4)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:27.520177 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:27.601894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf7c8bfc6b54015b03ebbeb3406e7a4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:27.614252 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:27.618267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf7c8bfc6b54015b03ebbeb3406e7a4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:27.619102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf7c8bfc6b54015b03ebbeb3406e7a4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:27.619673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf7c8bfc6b54015b03ebbeb3406e7a4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:27.641149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-063e3868-06cd-4920-b399-277381d6115b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf7c8bfc6b54015b03ebbeb3406e7a4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:27.653574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1201/2387] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:27 2022] DELETE /identity/v3/users/0bf7c8bfc6b54015b03ebbeb3406e7a4 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:28.208715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:28.210387 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:28.226990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe57220-29e6-45cb-a5eb-b8c10ca0e0de admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:28.298881 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:28.300538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:28.300856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:28.301288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] PATH_INFO: `/v3/projects/49d9d7af744b4558a0beac722bab3ffe` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:28.310623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d9d7af744b4558a0beac722bab3ffe)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:28.310623 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:28.387609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d9d7af744b4558a0beac722bab3ffe', 'request_id': 'req-ecbb02e1-6812-4b77-b492-0568b46c52ca'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:28.398251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d9d7af744b4558a0beac722bab3ffe', 'request_id': 'req-ecbb02e1-6812-4b77-b492-0568b46c52ca'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:28.405526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d9d7af744b4558a0beac722bab3ffe', 'request_id': 'req-ecbb02e1-6812-4b77-b492-0568b46c52ca'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:28.418264 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] The token cache is being invalidate because project 49d9d7af744b4558a0beac722bab3ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:28.418714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:28.419552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ecbb02e1-6812-4b77-b492-0568b46c52ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:28.421782 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1187/2388] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:28 2022] DELETE /identity/v3/projects/49d9d7af744b4558a0beac722bab3ffe => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:28.427762 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51a101-b116-4456-a0b4-705345e40744 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:28.428219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51a101-b116-4456-a0b4-705345e40744 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:28.428533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51a101-b116-4456-a0b4-705345e40744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:28.448226 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-aa51a101-b116-4456-a0b4-705345e40744 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:28.449619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-aa51a101-b116-4456-a0b4-705345e40744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:28.523855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1202/2389] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:28.533577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f852993-ea0a-4b7e-b658-efe45648071e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:28.533874 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f852993-ea0a-4b7e-b658-efe45648071e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:28.534196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f852993-ea0a-4b7e-b658-efe45648071e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:28.564583 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4f852993-ea0a-4b7e-b658-efe45648071e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:28.565876 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4f852993-ea0a-4b7e-b658-efe45648071e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:28.575398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1188/2390] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:28.579824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51a101-b116-4456-a0b4-705345e40744 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:28.590630 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51a101-b116-4456-a0b4-705345e40744 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:28.595884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d193e-b405-40d6-84c0-ec296c7de929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:28.596569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d193e-b405-40d6-84c0-ec296c7de929 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:28.596857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d193e-b405-40d6-84c0-ec296c7de929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:28.597123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d193e-b405-40d6-84c0-ec296c7de929 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:28.597864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d193e-b405-40d6-84c0-ec296c7de929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:07:28.598214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d193e-b405-40d6-84c0-ec296c7de929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:28.599868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d193e-b405-40d6-84c0-ec296c7de929 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:28.617318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1203/2391] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:07:28.618839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f852993-ea0a-4b7e-b658-efe45648071e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:28.632716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f852993-ea0a-4b7e-b658-efe45648071e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:28.641903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed0ad0-f5a7-4a6a-9876-83719af1020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:28.642647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed0ad0-f5a7-4a6a-9876-83719af1020d admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:28.642647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed0ad0-f5a7-4a6a-9876-83719af1020d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:28.642647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed0ad0-f5a7-4a6a-9876-83719af1020d admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:28.643466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed0ad0-f5a7-4a6a-9876-83719af1020d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:28.645533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed0ad0-f5a7-4a6a-9876-83719af1020d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:28.676886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:28.691585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1189/2392] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:28 2022] POST /identity/v3/projects => generated 393 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:28.700130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d193e-b405-40d6-84c0-ec296c7de929 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:28.726564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d193e-b405-40d6-84c0-ec296c7de929 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:28.868935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905d5a-b474-434f-bdfc-d2fb76ed2197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:28.869659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905d5a-b474-434f-bdfc-d2fb76ed2197 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:28.869941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905d5a-b474-434f-bdfc-d2fb76ed2197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:28.870246 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905d5a-b474-434f-bdfc-d2fb76ed2197 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:28.870964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905d5a-b474-434f-bdfc-d2fb76ed2197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:28.872925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905d5a-b474-434f-bdfc-d2fb76ed2197 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:28.924049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1204/2393] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:28 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:28.928644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed0ad0-f5a7-4a6a-9876-83719af1020d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:28.948632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed0ad0-f5a7-4a6a-9876-83719af1020d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:28.953607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463ab76-1fac-4fb6-ab4c-5c1c61b5908f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:28.954659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463ab76-1fac-4fb6-ab4c-5c1c61b5908f admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:28.955119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463ab76-1fac-4fb6-ab4c-5c1c61b5908f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:28.955398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463ab76-1fac-4fb6-ab4c-5c1c61b5908f admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:28.956209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463ab76-1fac-4fb6-ab4c-5c1c61b5908f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:28.957794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463ab76-1fac-4fb6-ab4c-5c1c61b5908f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:28.970001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1190/2394] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:28.974048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905d5a-b474-434f-bdfc-d2fb76ed2197 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:28.986241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905d5a-b474-434f-bdfc-d2fb76ed2197 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:28.997766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c93dce-7777-4cd3-997c-5e3070a9fc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:28.998907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c93dce-7777-4cd3-997c-5e3070a9fc2c admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:28.999199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c93dce-7777-4cd3-997c-5e3070a9fc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:28.999487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c93dce-7777-4cd3-997c-5e3070a9fc2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.000210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c93dce-7777-4cd3-997c-5e3070a9fc2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:29.001865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c93dce-7777-4cd3-997c-5e3070a9fc2c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:29.023787 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1205/2395] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:29.027107 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463ab76-1fac-4fb6-ab4c-5c1c61b5908f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:29.038441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463ab76-1fac-4fb6-ab4c-5c1c61b5908f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:29.043420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcb1b8-cf05-4937-9238-9d855a82f659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:29.044323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb1b8-cf05-4937-9238-9d855a82f659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:29.044609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb1b8-cf05-4937-9238-9d855a82f659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:29.044885 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb1b8-cf05-4937-9238-9d855a82f659 admin admin] PATH_INFO: `/v3/projects/a0f8b9c81bef4527bbcb9bf7ac5f0124/users/6352cd3715434b24b86d93f61196b7c8/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.060155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcb1b8-cf05-4937-9238-9d855a82f659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f8b9c81bef4527bbcb9bf7ac5f0124, user_id=6352cd3715434b24b86d93f61196b7c8, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:29.062975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcb1b8-cf05-4937-9238-9d855a82f659 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:29.077599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1191/2396] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:29 2022] PUT /identity/v3/projects/a0f8b9c81bef4527bbcb9bf7ac5f0124/users/6352cd3715434b24b86d93f61196b7c8/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:29.088886 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a113bfb-5813-44b6-bb3d-228d6e0623bb None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:29.089175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a113bfb-5813-44b6-bb3d-228d6e0623bb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:29.089449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a113bfb-5813-44b6-bb3d-228d6e0623bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.118387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0a113bfb-5813-44b6-bb3d-228d6e0623bb None None] MFA Rules not processed for user `6352cd3715434b24b86d93f61196b7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:29.119526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0a113bfb-5813-44b6-bb3d-228d6e0623bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:29.139168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:29.195213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1206/2397] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:29.199814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcb1b8-cf05-4937-9238-9d855a82f659 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:29.270848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcb1b8-cf05-4937-9238-9d855a82f659 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:29.275588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54700c-3cac-4d2b-9e30-b9e6e28dd3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:29.276327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54700c-3cac-4d2b-9e30-b9e6e28dd3de admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:29.276674 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54700c-3cac-4d2b-9e30-b9e6e28dd3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:29.277024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54700c-3cac-4d2b-9e30-b9e6e28dd3de admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.277783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54700c-3cac-4d2b-9e30-b9e6e28dd3de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:29.280103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54700c-3cac-4d2b-9e30-b9e6e28dd3de admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:29.315242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1192/2398] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:29 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:29.319331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a113bfb-5813-44b6-bb3d-228d6e0623bb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:29.333731 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a113bfb-5813-44b6-bb3d-228d6e0623bb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:29.349478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54700c-3cac-4d2b-9e30-b9e6e28dd3de admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:29.352490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0c54700c-3cac-4d2b-9e30-b9e6e28dd3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:29.465457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ee588-2e51-4edf-b0da-747a5087de22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:29.466179 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ee588-2e51-4edf-b0da-747a5087de22 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:29.466531 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ee588-2e51-4edf-b0da-747a5087de22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:29.466938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ee588-2e51-4edf-b0da-747a5087de22 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.467890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ee588-2e51-4edf-b0da-747a5087de22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:29.469932 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ee588-2e51-4edf-b0da-747a5087de22 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:29.518320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54700c-3cac-4d2b-9e30-b9e6e28dd3de admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:29.518666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1207/2399] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:29 2022] POST /identity/v3/users => generated 303 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:29.522268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ee588-2e51-4edf-b0da-747a5087de22 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:29.524576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451873d3-a18e-41d1-a239-f0a42a274791 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:29.525350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451873d3-a18e-41d1-a239-f0a42a274791 service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:29.525709 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451873d3-a18e-41d1-a239-f0a42a274791 service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:29.526047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451873d3-a18e-41d1-a239-f0a42a274791 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.528314 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-451873d3-a18e-41d1-a239-f0a42a274791 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:29.535925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ee588-2e51-4edf-b0da-747a5087de22 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:29.542874 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3f04b-fb74-4d7c-afce-d89366779cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:29.543395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451873d3-a18e-41d1-a239-f0a42a274791 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:29.544866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f04b-fb74-4d7c-afce-d89366779cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:29.545428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451873d3-a18e-41d1-a239-f0a42a274791 service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:29.545528 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f04b-fb74-4d7c-afce-d89366779cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:29.545632 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f04b-fb74-4d7c-afce-d89366779cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.546441 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3f04b-fb74-4d7c-afce-d89366779cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:29.549233 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3f04b-fb74-4d7c-afce-d89366779cc8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:29.568496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1208/2400] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:29.572402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1193/2401] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:29.573873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3f04b-fb74-4d7c-afce-d89366779cc8 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:29.588901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3f04b-fb74-4d7c-afce-d89366779cc8 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:29.594720 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba56b9-e025-4f55-81a5-3249f135c408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:29.595675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba56b9-e025-4f55-81a5-3249f135c408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:29.595958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba56b9-e025-4f55-81a5-3249f135c408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:29.596232 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba56b9-e025-4f55-81a5-3249f135c408 admin admin] PATH_INFO: `/v3/projects/1122c897fa734d2e9bc85131c99d2bd2/users/c86256b7f3494414b1427108de8f7127/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.620194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba56b9-e025-4f55-81a5-3249f135c408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1122c897fa734d2e9bc85131c99d2bd2, user_id=c86256b7f3494414b1427108de8f7127, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:29.624935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba56b9-e025-4f55-81a5-3249f135c408 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:29.651916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1209/2402] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:29 2022] PUT /identity/v3/projects/1122c897fa734d2e9bc85131c99d2bd2/users/c86256b7f3494414b1427108de8f7127/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:29.657719 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451873d3-a18e-41d1-a239-f0a42a274791 service placement] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:29.671538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451873d3-a18e-41d1-a239-f0a42a274791 service placement] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:29.686222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:29.754954 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf24391-64a1-4ae6-9e6a-36a9936bbdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:29.755647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24391-64a1-4ae6-9e6a-36a9936bbdcd admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:29.755960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24391-64a1-4ae6-9e6a-36a9936bbdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:29.756227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf24391-64a1-4ae6-9e6a-36a9936bbdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.757064 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf24391-64a1-4ae6-9e6a-36a9936bbdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:29.758826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf24391-64a1-4ae6-9e6a-36a9936bbdcd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:29.775041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1194/2403] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:29.776237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba56b9-e025-4f55-81a5-3249f135c408 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:29.789370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba56b9-e025-4f55-81a5-3249f135c408 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:29.794865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b069346-8be4-4c31-a4fa-acdb6a38f8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:29.795909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b069346-8be4-4c31-a4fa-acdb6a38f8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:29.796456 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b069346-8be4-4c31-a4fa-acdb6a38f8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:29.796805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b069346-8be4-4c31-a4fa-acdb6a38f8e6 admin admin] PATH_INFO: `/v3/projects/1122c897fa734d2e9bc85131c99d2bd2/users/c86256b7f3494414b1427108de8f7127/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.799037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b069346-8be4-4c31-a4fa-acdb6a38f8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1122c897fa734d2e9bc85131c99d2bd2, user_id=c86256b7f3494414b1427108de8f7127, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:29.802241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b069346-8be4-4c31-a4fa-acdb6a38f8e6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:29.815886 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1210/2404] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:29 2022] PUT /identity/v3/projects/1122c897fa734d2e9bc85131c99d2bd2/users/c86256b7f3494414b1427108de8f7127/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:29.835668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bbf75-5798-421e-b4af-586d8d75d686 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:29.836036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bbf75-5798-421e-b4af-586d8d75d686 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:29.836242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bbf75-5798-421e-b4af-586d8d75d686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:29.883594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b16bbf75-5798-421e-b4af-586d8d75d686 None None] MFA Rules not processed for user `c86256b7f3494414b1427108de8f7127`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:29.884816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b16bbf75-5798-421e-b4af-586d8d75d686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:30.030137 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1195/2405] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:30.043221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b069346-8be4-4c31-a4fa-acdb6a38f8e6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:30.046316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0b069346-8be4-4c31-a4fa-acdb6a38f8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:30.203059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:30.227584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b069346-8be4-4c31-a4fa-acdb6a38f8e6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:30.231744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2a134-51a8-4d42-9b98-a02c4cf255ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:30.232582 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2a134-51a8-4d42-9b98-a02c4cf255ab service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:30.232995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2a134-51a8-4d42-9b98-a02c4cf255ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:30.233339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2a134-51a8-4d42-9b98-a02c4cf255ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:30.243603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2a134-51a8-4d42-9b98-a02c4cf255ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:30.245190 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2a134-51a8-4d42-9b98-a02c4cf255ab service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:30.257205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1211/2406] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:07:30.313536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bbf75-5798-421e-b4af-586d8d75d686 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:30.324903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bbf75-5798-421e-b4af-586d8d75d686 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:30.329375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f75749-d037-4a9a-91ad-86c1fc27b9ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:30.329979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f75749-d037-4a9a-91ad-86c1fc27b9ef service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:30.330441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f75749-d037-4a9a-91ad-86c1fc27b9ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:30.330646 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f75749-d037-4a9a-91ad-86c1fc27b9ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:30.400700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f75749-d037-4a9a-91ad-86c1fc27b9ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:30.402349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f75749-d037-4a9a-91ad-86c1fc27b9ef service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:30.414042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1196/2407] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:33.428592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d214e-32d8-4e98-9902-884e5eea9ed4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:33.429114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d214e-32d8-4e98-9902-884e5eea9ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:33.429215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d214e-32d8-4e98-9902-884e5eea9ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:33.468812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b67d214e-32d8-4e98-9902-884e5eea9ed4 None None] MFA Rules not processed for user `fdff9a743af948f88d60a6fd80795c06`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:33.470392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b67d214e-32d8-4e98-9902-884e5eea9ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:33.552916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1212/2408] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:33.557488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f75749-d037-4a9a-91ad-86c1fc27b9ef service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:33.559126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-69f75749-d037-4a9a-91ad-86c1fc27b9ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:33.619464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:33.644485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f75749-d037-4a9a-91ad-86c1fc27b9ef service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:33.649072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d330e5-b34a-418a-bcff-507d496ef0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:33.649774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d330e5-b34a-418a-bcff-507d496ef0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:33.650053 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d330e5-b34a-418a-bcff-507d496ef0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:33.650385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d330e5-b34a-418a-bcff-507d496ef0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:33.651080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d330e5-b34a-418a-bcff-507d496ef0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:33.653069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d330e5-b34a-418a-bcff-507d496ef0b4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:33.680384 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1197/2409] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:33 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:33.684564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d214e-32d8-4e98-9902-884e5eea9ed4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:33.697327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d214e-32d8-4e98-9902-884e5eea9ed4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:33.768301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436330b1-1b5a-4b46-905f-c4e55e92a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:33.769022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436330b1-1b5a-4b46-905f-c4e55e92a651 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:33.769409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436330b1-1b5a-4b46-905f-c4e55e92a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:33.769757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436330b1-1b5a-4b46-905f-c4e55e92a651 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:33.770641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436330b1-1b5a-4b46-905f-c4e55e92a651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:33.772653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436330b1-1b5a-4b46-905f-c4e55e92a651 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:33.800722 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1213/2410] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:33 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:07:33.801912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d330e5-b34a-418a-bcff-507d496ef0b4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:33.813995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d330e5-b34a-418a-bcff-507d496ef0b4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:33.819079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e0c04-5d30-4f20-8095-d424ccb7d3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:33.819759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e0c04-5d30-4f20-8095-d424ccb7d3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:33.820041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e0c04-5d30-4f20-8095-d424ccb7d3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:33.820336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e0c04-5d30-4f20-8095-d424ccb7d3d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:33.821072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e0c04-5d30-4f20-8095-d424ccb7d3d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:33.822771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e0c04-5d30-4f20-8095-d424ccb7d3d0 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:33.837441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1198/2411] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:33.841646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436330b1-1b5a-4b46-905f-c4e55e92a651 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:33.852145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436330b1-1b5a-4b46-905f-c4e55e92a651 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:33.857222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d108c-fad0-4a8d-bf93-539a7b9a7325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:33.858496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d108c-fad0-4a8d-bf93-539a7b9a7325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:33.858778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d108c-fad0-4a8d-bf93-539a7b9a7325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:33.859050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d108c-fad0-4a8d-bf93-539a7b9a7325 admin admin] PATH_INFO: `/v3/projects/e7a519fd2119461cb1985e6d3752b212/users/4ea5ff9a8c45438f980d1b1755e5b815/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:33.877121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d108c-fad0-4a8d-bf93-539a7b9a7325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a519fd2119461cb1985e6d3752b212, user_id=4ea5ff9a8c45438f980d1b1755e5b815, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:33.879860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d108c-fad0-4a8d-bf93-539a7b9a7325 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:33.897189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1214/2412] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:33 2022] PUT /identity/v3/projects/e7a519fd2119461cb1985e6d3752b212/users/4ea5ff9a8c45438f980d1b1755e5b815/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:33.901145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e0c04-5d30-4f20-8095-d424ccb7d3d0 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:33.912853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e0c04-5d30-4f20-8095-d424ccb7d3d0 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:33.975569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e0276-472c-4658-9431-e60ea4f9f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:33.976244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0276-472c-4658-9431-e60ea4f9f439 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:33.976520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0276-472c-4658-9431-e60ea4f9f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:33.976791 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e0276-472c-4658-9431-e60ea4f9f439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:33.977518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e0276-472c-4658-9431-e60ea4f9f439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:33.979171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e0276-472c-4658-9431-e60ea4f9f439 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:33.989590 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1199/2413] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:33.994147 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d108c-fad0-4a8d-bf93-539a7b9a7325 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:34.006235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d108c-fad0-4a8d-bf93-539a7b9a7325 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:34.010846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7a863-58b9-4924-8270-910859058b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:34.011738 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7a863-58b9-4924-8270-910859058b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:34.012049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7a863-58b9-4924-8270-910859058b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:34.012322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7a863-58b9-4924-8270-910859058b4e admin admin] PATH_INFO: `/v3/projects/e7a519fd2119461cb1985e6d3752b212/users/4ea5ff9a8c45438f980d1b1755e5b815/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:34.014490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7a863-58b9-4924-8270-910859058b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a519fd2119461cb1985e6d3752b212, user_id=4ea5ff9a8c45438f980d1b1755e5b815, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:34.017234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7a863-58b9-4924-8270-910859058b4e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:34.027504 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1215/2414] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:33 2022] PUT /identity/v3/projects/e7a519fd2119461cb1985e6d3752b212/users/4ea5ff9a8c45438f980d1b1755e5b815/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:37.518802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e0276-472c-4658-9431-e60ea4f9f439 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:37.536164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e0276-472c-4658-9431-e60ea4f9f439 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:37.682928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:37.684145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:37.684581 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:37.685052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] PATH_INFO: `/v3/users/6352cd3715434b24b86d93f61196b7c8` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:37.689785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6352cd3715434b24b86d93f61196b7c8)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:37.695850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:37.813216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6352cd3715434b24b86d93f61196b7c8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:37.814297 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6352cd3715434b24b86d93f61196b7c8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:37.820899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:37.839861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6352cd3715434b24b86d93f61196b7c8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:37.841551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6352cd3715434b24b86d93f61196b7c8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:37.870329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6352cd3715434b24b86d93f61196b7c8'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:37.944752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1200/2415] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:37 2022] DELETE /identity/v3/users/6352cd3715434b24b86d93f61196b7c8 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:37.949377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128f90e-95f7-4227-8d69-240cfe7810c8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:37.949876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128f90e-95f7-4227-8d69-240cfe7810c8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:37.950252 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128f90e-95f7-4227-8d69-240cfe7810c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:37.990575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9128f90e-95f7-4227-8d69-240cfe7810c8 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:37.995510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9128f90e-95f7-4227-8d69-240cfe7810c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:38.105879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:38.237022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1216/2416] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:38.248355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:38.250642 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:38.352508 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04785f83-3871-48f9-9da1-afcbfd68a395 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:38.357251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e3446-0e69-4cbe-aeaa-8a1194804d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:38.358690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3446-0e69-4cbe-aeaa-8a1194804d73 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:38.359009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3446-0e69-4cbe-aeaa-8a1194804d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:38.359255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e3446-0e69-4cbe-aeaa-8a1194804d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:38.371706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e3446-0e69-4cbe-aeaa-8a1194804d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:38.373253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e3446-0e69-4cbe-aeaa-8a1194804d73 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:38.384613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1201/2417] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:38.757205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128f90e-95f7-4227-8d69-240cfe7810c8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:38.760133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9128f90e-95f7-4227-8d69-240cfe7810c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:38.774520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128f90e-95f7-4227-8d69-240cfe7810c8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:38.781726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:38.782947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:38.783225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:38.783658 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] PATH_INFO: `/v3/projects/a0f8b9c81bef4527bbcb9bf7ac5f0124` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:38.784887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f8b9c81bef4527bbcb9bf7ac5f0124)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:38.786849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:38.862283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f8b9c81bef4527bbcb9bf7ac5f0124', 'request_id': 'req-a3bc5111-da5d-4784-9df4-1005582d6fe6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:38.878268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f8b9c81bef4527bbcb9bf7ac5f0124', 'request_id': 'req-a3bc5111-da5d-4784-9df4-1005582d6fe6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:38.902397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f8b9c81bef4527bbcb9bf7ac5f0124', 'request_id': 'req-a3bc5111-da5d-4784-9df4-1005582d6fe6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:38.932071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] The token cache is being invalidate because project a0f8b9c81bef4527bbcb9bf7ac5f0124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:38.932071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:38.932619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:38.937477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1217/2418] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:38 2022] DELETE /identity/v3/projects/a0f8b9c81bef4527bbcb9bf7ac5f0124 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:38.939064 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e3446-0e69-4cbe-aeaa-8a1194804d73 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:38.954771 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e3446-0e69-4cbe-aeaa-8a1194804d73 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:39.069919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:39.070929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:39.071194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:39.071476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] PATH_INFO: `/v3/users/c86256b7f3494414b1427108de8f7127` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:39.072759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c86256b7f3494414b1427108de8f7127)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:39.074770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:39.143381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c86256b7f3494414b1427108de8f7127'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:39.143999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c86256b7f3494414b1427108de8f7127'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:39.167427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c86256b7f3494414b1427108de8f7127'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:39.169051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c86256b7f3494414b1427108de8f7127'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:39.181927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a0689fe-1379-4482-b48d-f522472d6c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c86256b7f3494414b1427108de8f7127'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:39.190002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:39.208277 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1202/2419] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:38 2022] DELETE /identity/v3/users/c86256b7f3494414b1427108de8f7127 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:39.730984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:39.732914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:39.758926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc5111-da5d-4784-9df4-1005582d6fe6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:39.899708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:39.900824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:39.901449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:39.907261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] PATH_INFO: `/v3/projects/1122c897fa734d2e9bc85131c99d2bd2` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:39.910286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=1122c897fa734d2e9bc85131c99d2bd2)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:39.912603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:40.003907 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1122c897fa734d2e9bc85131c99d2bd2', 'request_id': 'req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:40.008012 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:40.026428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1122c897fa734d2e9bc85131c99d2bd2', 'request_id': 'req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:40.027641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b119fc-4d88-4b55-a153-28c1b2c95785 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.028093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b119fc-4d88-4b55-a153-28c1b2c95785 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.028250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b119fc-4d88-4b55-a153-28c1b2c95785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.034280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1122c897fa734d2e9bc85131c99d2bd2', 'request_id': 'req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:40.051998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] The token cache is being invalidate because project 1122c897fa734d2e9bc85131c99d2bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:40.053889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:40.054638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fa7f26d6-42f2-4609-b8d2-7888ed8d15ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:40.056849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1218/2420] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:39 2022] DELETE /identity/v3/projects/1122c897fa734d2e9bc85131c99d2bd2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:40.063102 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829431b-52dd-4316-85f5-4ae2c607b2ee None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.063471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829431b-52dd-4316-85f5-4ae2c607b2ee None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.063988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829431b-52dd-4316-85f5-4ae2c607b2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.065117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-08b119fc-4d88-4b55-a153-28c1b2c95785 None None] MFA Rules not processed for user `4ea5ff9a8c45438f980d1b1755e5b815`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:40.066416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-08b119fc-4d88-4b55-a153-28c1b2c95785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:40.090648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7829431b-52dd-4316-85f5-4ae2c607b2ee None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:40.092365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7829431b-52dd-4316-85f5-4ae2c607b2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:40.169492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1203/2421] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:40.177127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1219/2422] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:40.178558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9614f-d1c6-454a-acba-2371fb4749c2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.178850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9614f-d1c6-454a-acba-2371fb4749c2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.179132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9614f-d1c6-454a-acba-2371fb4749c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.183233 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829431b-52dd-4316-85f5-4ae2c607b2ee None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:40.184608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7829431b-52dd-4316-85f5-4ae2c607b2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:40.206329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-14f9614f-d1c6-454a-acba-2371fb4749c2 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:40.208218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-14f9614f-d1c6-454a-acba-2371fb4749c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:40.223169 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1204/2423] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:40.227296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9614f-d1c6-454a-acba-2371fb4749c2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:40.237168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9614f-d1c6-454a-acba-2371fb4749c2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:40.242538 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007964c7-ee59-41f5-9cae-f7c16d41b2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:40.243681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007964c7-ee59-41f5-9cae-f7c16d41b2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.243681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007964c7-ee59-41f5-9cae-f7c16d41b2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.243922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007964c7-ee59-41f5-9cae-f7c16d41b2b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.244483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007964c7-ee59-41f5-9cae-f7c16d41b2b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:07:40.244793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007964c7-ee59-41f5-9cae-f7c16d41b2b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:40.246473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007964c7-ee59-41f5-9cae-f7c16d41b2b9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:40.259649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1205/2424] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:07:40.264132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007964c7-ee59-41f5-9cae-f7c16d41b2b9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:40.276482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007964c7-ee59-41f5-9cae-f7c16d41b2b9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:40.282836 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f1b85-964e-4eab-a033-7fe468f6be1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:40.286133 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f1b85-964e-4eab-a033-7fe468f6be1c admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.286641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f1b85-964e-4eab-a033-7fe468f6be1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.286921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f1b85-964e-4eab-a033-7fe468f6be1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.287638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f1b85-964e-4eab-a033-7fe468f6be1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:40.289726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f1b85-964e-4eab-a033-7fe468f6be1c admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:40.314286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829431b-52dd-4316-85f5-4ae2c607b2ee None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:40.320875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84557a22-b016-44e1-8a68-c095f22de4e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:40.321537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84557a22-b016-44e1-8a68-c095f22de4e5 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.321842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84557a22-b016-44e1-8a68-c095f22de4e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.325067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84557a22-b016-44e1-8a68-c095f22de4e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.325206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1206/2425] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:40 2022] POST /identity/v3/projects => generated 399 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:40.325317 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-84557a22-b016-44e1-8a68-c095f22de4e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:40.326511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f1b85-964e-4eab-a033-7fe468f6be1c admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:40.388262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:40.404495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84557a22-b016-44e1-8a68-c095f22de4e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:40.408737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84557a22-b016-44e1-8a68-c095f22de4e5 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:40.410566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f1b85-964e-4eab-a033-7fe468f6be1c admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:40.425497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1220/2426] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:40.436636 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee283c-7bdf-40b2-bc68-6f470e778710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:40.437346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee283c-7bdf-40b2-bc68-6f470e778710 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.437630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee283c-7bdf-40b2-bc68-6f470e778710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.437906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee283c-7bdf-40b2-bc68-6f470e778710 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.438666 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee283c-7bdf-40b2-bc68-6f470e778710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:40.440635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee283c-7bdf-40b2-bc68-6f470e778710 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:40.487664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1207/2427] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:40 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:40.492152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84557a22-b016-44e1-8a68-c095f22de4e5 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:40.510386 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84557a22-b016-44e1-8a68-c095f22de4e5 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:40.521996 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6f08-568e-461b-9ed1-9d4e6ddad90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:40.523987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6f08-568e-461b-9ed1-9d4e6ddad90c admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.524500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6f08-568e-461b-9ed1-9d4e6ddad90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.525209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6f08-568e-461b-9ed1-9d4e6ddad90c admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.526531 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b6f08-568e-461b-9ed1-9d4e6ddad90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:40.529148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b6f08-568e-461b-9ed1-9d4e6ddad90c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:40.550833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1221/2428] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:40.555454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee283c-7bdf-40b2-bc68-6f470e778710 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:40.572404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee283c-7bdf-40b2-bc68-6f470e778710 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:40.583604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0b0b0-896b-4232-a546-b7aeabdd5dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:40.584467 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0b0b0-896b-4232-a546-b7aeabdd5dde admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.584746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0b0b0-896b-4232-a546-b7aeabdd5dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.585023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0b0b0-896b-4232-a546-b7aeabdd5dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.585768 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0b0b0-896b-4232-a546-b7aeabdd5dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:40.587471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0b0b0-896b-4232-a546-b7aeabdd5dde admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:40.604129 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1208/2429] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:40.605800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6f08-568e-461b-9ed1-9d4e6ddad90c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:40.624435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6f08-568e-461b-9ed1-9d4e6ddad90c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:40.638567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c75e72-1b37-451f-aaac-1eda8d72ca0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:40.639507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c75e72-1b37-451f-aaac-1eda8d72ca0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.639793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c75e72-1b37-451f-aaac-1eda8d72ca0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.640968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c75e72-1b37-451f-aaac-1eda8d72ca0a admin admin] PATH_INFO: `/v3/projects/3559a75fba2f40ccafb9a0f3b7bafbe1/users/dfdeaedb29ec4dc0a8c6b63846979035/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.665728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c75e72-1b37-451f-aaac-1eda8d72ca0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3559a75fba2f40ccafb9a0f3b7bafbe1, user_id=dfdeaedb29ec4dc0a8c6b63846979035, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:40.670108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c75e72-1b37-451f-aaac-1eda8d72ca0a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:40.706140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1222/2430] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:40 2022] PUT /identity/v3/projects/3559a75fba2f40ccafb9a0f3b7bafbe1/users/dfdeaedb29ec4dc0a8c6b63846979035/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:40.711783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a9d75-3741-4820-833a-caef59db2a84 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.712075 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a9d75-3741-4820-833a-caef59db2a84 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.712352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a9d75-3741-4820-833a-caef59db2a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.754851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:40.773689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d64a9d75-3741-4820-833a-caef59db2a84 None None] MFA Rules not processed for user `dfdeaedb29ec4dc0a8c6b63846979035`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:40.777409 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d64a9d75-3741-4820-833a-caef59db2a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:40.885064 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1209/2431] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:40.889690 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c75e72-1b37-451f-aaac-1eda8d72ca0a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:40.965750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c75e72-1b37-451f-aaac-1eda8d72ca0a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:40.971545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06160bb0-f467-4eb5-9503-daf890908ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:40.971990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06160bb0-f467-4eb5-9503-daf890908ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:40.972273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06160bb0-f467-4eb5-9503-daf890908ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:40.972548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06160bb0-f467-4eb5-9503-daf890908ed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:40.973249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06160bb0-f467-4eb5-9503-daf890908ed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:40.975205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06160bb0-f467-4eb5-9503-daf890908ed8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:41.003879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1223/2432] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:40 2022] POST /identity/v3/projects => generated 401 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:07:41.005030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a9d75-3741-4820-833a-caef59db2a84 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:41.024502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a9d75-3741-4820-833a-caef59db2a84 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:41.088675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c18ef-fd38-45ac-93a5-8664222b1226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:41.089392 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c18ef-fd38-45ac-93a5-8664222b1226 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:41.089633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c18ef-fd38-45ac-93a5-8664222b1226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:41.089905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c18ef-fd38-45ac-93a5-8664222b1226 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:41.090634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c18ef-fd38-45ac-93a5-8664222b1226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:41.092604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c18ef-fd38-45ac-93a5-8664222b1226 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:41.114009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1210/2433] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:41 2022] POST /identity/v3/users => generated 307 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:07:41.118981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06160bb0-f467-4eb5-9503-daf890908ed8 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:41.132706 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06160bb0-f467-4eb5-9503-daf890908ed8 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:41.138444 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3ce9a-f0b2-4e55-961a-9c9e0f587ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:41.139304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3ce9a-f0b2-4e55-961a-9c9e0f587ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:41.139687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3ce9a-f0b2-4e55-961a-9c9e0f587ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:41.140088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3ce9a-f0b2-4e55-961a-9c9e0f587ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:41.141235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3ce9a-f0b2-4e55-961a-9c9e0f587ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:41.143114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3ce9a-f0b2-4e55-961a-9c9e0f587ef1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:41.161961 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c18ef-fd38-45ac-93a5-8664222b1226 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:41.162591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1224/2434] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:41.174227 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c18ef-fd38-45ac-93a5-8664222b1226 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:41.179293 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b529eb2-288d-4dee-9a7a-d0a353d492bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:41.180379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b529eb2-288d-4dee-9a7a-d0a353d492bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:41.180851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b529eb2-288d-4dee-9a7a-d0a353d492bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:41.181252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b529eb2-288d-4dee-9a7a-d0a353d492bd admin admin] PATH_INFO: `/v3/projects/ce20193762824523845b1e32dee0e726/users/4effd461757841418964019a2bca194f/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:41.208863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b529eb2-288d-4dee-9a7a-d0a353d492bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce20193762824523845b1e32dee0e726, user_id=4effd461757841418964019a2bca194f, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:41.211723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b529eb2-288d-4dee-9a7a-d0a353d492bd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:41.235613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1211/2435] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:41 2022] PUT /identity/v3/projects/ce20193762824523845b1e32dee0e726/users/4effd461757841418964019a2bca194f/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:41.243035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3ce9a-f0b2-4e55-961a-9c9e0f587ef1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:41.260540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3ce9a-f0b2-4e55-961a-9c9e0f587ef1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:41.347489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce561c-f23e-42a3-8cf6-0fa04546822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:41.348217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce561c-f23e-42a3-8cf6-0fa04546822b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:41.348507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce561c-f23e-42a3-8cf6-0fa04546822b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:41.348787 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce561c-f23e-42a3-8cf6-0fa04546822b admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:41.349719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce561c-f23e-42a3-8cf6-0fa04546822b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:41.351289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce561c-f23e-42a3-8cf6-0fa04546822b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:41.372125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1225/2436] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:41.376400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b529eb2-288d-4dee-9a7a-d0a353d492bd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:41.388908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b529eb2-288d-4dee-9a7a-d0a353d492bd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:41.393973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8f504-24a2-43f0-a978-e06f908abaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:41.395063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f504-24a2-43f0-a978-e06f908abaec admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:41.395418 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f504-24a2-43f0-a978-e06f908abaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:41.395770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f504-24a2-43f0-a978-e06f908abaec admin admin] PATH_INFO: `/v3/projects/ce20193762824523845b1e32dee0e726/users/4effd461757841418964019a2bca194f/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:41.397871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8f504-24a2-43f0-a978-e06f908abaec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce20193762824523845b1e32dee0e726, user_id=4effd461757841418964019a2bca194f, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:41.400639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8f504-24a2-43f0-a978-e06f908abaec admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:41.421087 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1212/2437] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:41 2022] PUT /identity/v3/projects/ce20193762824523845b1e32dee0e726/users/4effd461757841418964019a2bca194f/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:07:41.485715 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74bce-7493-4ab6-9785-da79fe2e8c5a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:41.486035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74bce-7493-4ab6-9785-da79fe2e8c5a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:41.486365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74bce-7493-4ab6-9785-da79fe2e8c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:41.524294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-79f74bce-7493-4ab6-9785-da79fe2e8c5a None None] MFA Rules not processed for user `4effd461757841418964019a2bca194f`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:41.525756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-79f74bce-7493-4ab6-9785-da79fe2e8c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:41.542965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:41.607998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1226/2438] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:41 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:41.618897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8f504-24a2-43f0-a978-e06f908abaec admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:41.620910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-71d8f504-24a2-43f0-a978-e06f908abaec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:41.682421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8f504-24a2-43f0-a978-e06f908abaec admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:41.686131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3986f18-7e82-463d-a405-942c6a71ac07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:41.686860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3986f18-7e82-463d-a405-942c6a71ac07 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:41.687143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3986f18-7e82-463d-a405-942c6a71ac07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:41.687419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3986f18-7e82-463d-a405-942c6a71ac07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:41.695092 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:41.697163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3986f18-7e82-463d-a405-942c6a71ac07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:41.699093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3986f18-7e82-463d-a405-942c6a71ac07 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:41.710958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1213/2439] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:41 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:41.802483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f74bce-7493-4ab6-9785-da79fe2e8c5a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:41.812217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f74bce-7493-4ab6-9785-da79fe2e8c5a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:41.816380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060063e-586b-45a4-8e55-fdf34c69f3a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:41.817116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060063e-586b-45a4-8e55-fdf34c69f3a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:41.817821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060063e-586b-45a4-8e55-fdf34c69f3a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:41.818181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060063e-586b-45a4-8e55-fdf34c69f3a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:41.885063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f060063e-586b-45a4-8e55-fdf34c69f3a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:41.886752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f060063e-586b-45a4-8e55-fdf34c69f3a4 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:41.897228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d5e80-7200-4047-a061-701f6059b25d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:41.897510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d5e80-7200-4047-a061-701f6059b25d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:41.897805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d5e80-7200-4047-a061-701f6059b25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:41.907030 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1227/2440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:41.950020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d06d5e80-7200-4047-a061-701f6059b25d None None] MFA Rules not processed for user `fdff9a743af948f88d60a6fd80795c06`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:41.951418 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d06d5e80-7200-4047-a061-701f6059b25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:42.074136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1214/2441] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:51.900860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=50 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:51.918146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060063e-586b-45a4-8e55-fdf34c69f3a4 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:51.919916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f060063e-586b-45a4-8e55-fdf34c69f3a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:51.991983 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060063e-586b-45a4-8e55-fdf34c69f3a4 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:51.997887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:51.998860 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:51.999226 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:51.999544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] PATH_INFO: `/v3/users/1d254e641c3f473b99b89ca4554b7f3a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:52.001412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d254e641c3f473b99b89ca4554b7f3a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:52.003413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:52.051248 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=27 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:52.061244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d254e641c3f473b99b89ca4554b7f3a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.070905 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d254e641c3f473b99b89ca4554b7f3a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.071605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d254e641c3f473b99b89ca4554b7f3a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.072317 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d254e641c3f473b99b89ca4554b7f3a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.094736 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d254e641c3f473b99b89ca4554b7f3a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.110465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1228/2442] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:51 2022] DELETE /identity/v3/users/1d254e641c3f473b99b89ca4554b7f3a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:52.115279 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d01d3-c39d-41a1-b43b-cca15836a554 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:52.115600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d01d3-c39d-41a1-b43b-cca15836a554 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:52.115965 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d01d3-c39d-41a1-b43b-cca15836a554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:52.135773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-fd9d01d3-c39d-41a1-b43b-cca15836a554 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:52.137023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fd9d01d3-c39d-41a1-b43b-cca15836a554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:52.203412 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1215/2443] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:52.214391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:52.215935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:52.279222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7e13a-fd86-41c9-a650-56fc0083956e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:52.283621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ed63f-e507-4538-b4db-5b59ebf1fd7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:52.284236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ed63f-e507-4538-b4db-5b59ebf1fd7f service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:52.284529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ed63f-e507-4538-b4db-5b59ebf1fd7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:52.284806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ed63f-e507-4538-b4db-5b59ebf1fd7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:52.294793 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ed63f-e507-4538-b4db-5b59ebf1fd7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:52.296231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ed63f-e507-4538-b4db-5b59ebf1fd7f service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:52.306436 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1229/2444] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:52.628846 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d01d3-c39d-41a1-b43b-cca15836a554 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:52.630295 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fd9d01d3-c39d-41a1-b43b-cca15836a554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:52.643614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d01d3-c39d-41a1-b43b-cca15836a554 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:52.648926 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:52.649993 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:52.650501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:52.650782 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] PATH_INFO: `/v3/users/dfdeaedb29ec4dc0a8c6b63846979035` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:52.651971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfdeaedb29ec4dc0a8c6b63846979035)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:52.653993 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:52.667671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ed63f-e507-4538-b4db-5b59ebf1fd7f service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:52.669681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3f5ed63f-e507-4538-b4db-5b59ebf1fd7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:52.686300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ed63f-e507-4538-b4db-5b59ebf1fd7f service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:52.694078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:52.695007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:52.695313 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:52.695562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] PATH_INFO: `/v3/projects/73b120df0997427bbdaedb45b7da1968` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:52.701094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=73b120df0997427bbdaedb45b7da1968)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:52.704287 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:52.723494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfdeaedb29ec4dc0a8c6b63846979035'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.724671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfdeaedb29ec4dc0a8c6b63846979035'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.734201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:52.736151 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfdeaedb29ec4dc0a8c6b63846979035'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.738496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfdeaedb29ec4dc0a8c6b63846979035'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.748186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a3e927b-38a4-40ac-97f2-f26827d50c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfdeaedb29ec4dc0a8c6b63846979035'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.772806 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1216/2445] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:52 2022] DELETE /identity/v3/users/dfdeaedb29ec4dc0a8c6b63846979035 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:52.777329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a52094-d268-4681-9ae4-b02486fdd7b1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:52.778543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a52094-d268-4681-9ae4-b02486fdd7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:52.779141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a52094-d268-4681-9ae4-b02486fdd7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:52.789708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73b120df0997427bbdaedb45b7da1968', 'request_id': 'req-3f180305-5c00-49ff-a438-f5bf339c81bd'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.800433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-69a52094-d268-4681-9ae4-b02486fdd7b1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:52.801691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-69a52094-d268-4681-9ae4-b02486fdd7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:52.802611 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73b120df0997427bbdaedb45b7da1968', 'request_id': 'req-3f180305-5c00-49ff-a438-f5bf339c81bd'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.812731 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73b120df0997427bbdaedb45b7da1968', 'request_id': 'req-3f180305-5c00-49ff-a438-f5bf339c81bd'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.828538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] The token cache is being invalidate because project 73b120df0997427bbdaedb45b7da1968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:52.828881 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.830378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3f180305-5c00-49ff-a438-f5bf339c81bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:52.833187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1230/2446] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:52 2022] DELETE /identity/v3/projects/73b120df0997427bbdaedb45b7da1968 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:07:52.838268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d811e1-5909-433c-bad9-8fd875b5155b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:52.838639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d811e1-5909-433c-bad9-8fd875b5155b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:52.838855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d811e1-5909-433c-bad9-8fd875b5155b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:52.862028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b6d811e1-5909-433c-bad9-8fd875b5155b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:52.863284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b6d811e1-5909-433c-bad9-8fd875b5155b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:52.877113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1217/2447] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:52.886613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a52094-d268-4681-9ae4-b02486fdd7b1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:52.888270 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-69a52094-d268-4681-9ae4-b02486fdd7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:52.903825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:52.942436 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1231/2448] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:52.949348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a0cf-7db5-420d-afbd-a74c06f88bc0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:52.949707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a0cf-7db5-420d-afbd-a74c06f88bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:52.949993 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a0cf-7db5-420d-afbd-a74c06f88bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:52.958341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a52094-d268-4681-9ae4-b02486fdd7b1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:52.963247 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8629b-1e11-47bc-95f2-291c177f742a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:52.963905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8629b-1e11-47bc-95f2-291c177f742a service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:52.964187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8629b-1e11-47bc-95f2-291c177f742a service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:52.964462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8629b-1e11-47bc-95f2-291c177f742a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:52.973729 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8c99a0cf-7db5-420d-afbd-a74c06f88bc0 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:52.975151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8c99a0cf-7db5-420d-afbd-a74c06f88bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:52.975276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8629b-1e11-47bc-95f2-291c177f742a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:52.977236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8629b-1e11-47bc-95f2-291c177f742a service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:52.984914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1232/2449] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:52.989593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a0cf-7db5-420d-afbd-a74c06f88bc0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:52.991851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1218/2450] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:07:52.999041 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a0cf-7db5-420d-afbd-a74c06f88bc0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:53.004258 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c06b41-32d1-4f1b-8213-9302250805fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:53.005094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c06b41-32d1-4f1b-8213-9302250805fd admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.005481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c06b41-32d1-4f1b-8213-9302250805fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.005840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c06b41-32d1-4f1b-8213-9302250805fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.006966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c06b41-32d1-4f1b-8213-9302250805fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:07:53.007073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c06b41-32d1-4f1b-8213-9302250805fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:53.008852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c06b41-32d1-4f1b-8213-9302250805fd admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:53.023521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1233/2451] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:07:53.027720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8629b-1e11-47bc-95f2-291c177f742a service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:53.040668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8629b-1e11-47bc-95f2-291c177f742a service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:53.049475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b9772-10b1-4fbf-8afd-9c5e5e29bee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:53.050263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b9772-10b1-4fbf-8afd-9c5e5e29bee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.050570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b9772-10b1-4fbf-8afd-9c5e5e29bee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.050835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b9772-10b1-4fbf-8afd-9c5e5e29bee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.051546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b9772-10b1-4fbf-8afd-9c5e5e29bee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:53.053476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b9772-10b1-4fbf-8afd-9c5e5e29bee2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:53.089837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1219/2452] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:53 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:07:53.093689 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c06b41-32d1-4f1b-8213-9302250805fd admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:53.112463 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c06b41-32d1-4f1b-8213-9302250805fd admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:53.208152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87f15d-999f-4053-9517-5f7eb4f0bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:53.208848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87f15d-999f-4053-9517-5f7eb4f0bcaa admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.209139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87f15d-999f-4053-9517-5f7eb4f0bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.209408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87f15d-999f-4053-9517-5f7eb4f0bcaa admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.210152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87f15d-999f-4053-9517-5f7eb4f0bcaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:53.212095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87f15d-999f-4053-9517-5f7eb4f0bcaa admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:53.237473 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1234/2453] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:53 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:07:53.242967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b9772-10b1-4fbf-8afd-9c5e5e29bee2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:53.254857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b9772-10b1-4fbf-8afd-9c5e5e29bee2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:53.259693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ec519-3b63-4500-8b2d-ef768392937f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:53.260389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ec519-3b63-4500-8b2d-ef768392937f admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.260677 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ec519-3b63-4500-8b2d-ef768392937f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.260959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ec519-3b63-4500-8b2d-ef768392937f admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.261707 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ec519-3b63-4500-8b2d-ef768392937f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:53.263371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ec519-3b63-4500-8b2d-ef768392937f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:53.275016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1220/2454] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:53.279185 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87f15d-999f-4053-9517-5f7eb4f0bcaa admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:53.296348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87f15d-999f-4053-9517-5f7eb4f0bcaa admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:53.301914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba140c20-a4b0-4f79-95b5-45168becee71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:53.302859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba140c20-a4b0-4f79-95b5-45168becee71 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.303382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba140c20-a4b0-4f79-95b5-45168becee71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.303827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba140c20-a4b0-4f79-95b5-45168becee71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.304905 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba140c20-a4b0-4f79-95b5-45168becee71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:53.307474 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba140c20-a4b0-4f79-95b5-45168becee71 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:53.327020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1235/2455] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:53.327327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ec519-3b63-4500-8b2d-ef768392937f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:53.329135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba140c20-a4b0-4f79-95b5-45168becee71 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:53.333922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ba140c20-a4b0-4f79-95b5-45168becee71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:53.339285 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ec519-3b63-4500-8b2d-ef768392937f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:53.345497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82db335-1b1c-4881-bd06-4b5cdcadd506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:53.346249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:53.346444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82db335-1b1c-4881-bd06-4b5cdcadd506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.346980 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82db335-1b1c-4881-bd06-4b5cdcadd506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.346980 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82db335-1b1c-4881-bd06-4b5cdcadd506 admin admin] PATH_INFO: `/v3/projects/61712d6374004516aaadcf07428b6c77/users/a270e9942e5840e094943db12eb10ce2/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.352059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba140c20-a4b0-4f79-95b5-45168becee71 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:53.358081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:53.360275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.361408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.362206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] PATH_INFO: `/v3/projects/3559a75fba2f40ccafb9a0f3b7bafbe1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.364547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3559a75fba2f40ccafb9a0f3b7bafbe1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:53.367239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:53.378300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82db335-1b1c-4881-bd06-4b5cdcadd506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61712d6374004516aaadcf07428b6c77, user_id=a270e9942e5840e094943db12eb10ce2, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:53.382039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82db335-1b1c-4881-bd06-4b5cdcadd506 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:53.398634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1221/2456] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:53 2022] PUT /identity/v3/projects/61712d6374004516aaadcf07428b6c77/users/a270e9942e5840e094943db12eb10ce2/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:53.411584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d60512-1146-4366-87bb-60c71178ec4e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.411796 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d60512-1146-4366-87bb-60c71178ec4e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.412048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d60512-1146-4366-87bb-60c71178ec4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.429922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3559a75fba2f40ccafb9a0f3b7bafbe1', 'request_id': 'req-723110e3-6c68-44da-95d6-147a18db39a1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.438823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3559a75fba2f40ccafb9a0f3b7bafbe1', 'request_id': 'req-723110e3-6c68-44da-95d6-147a18db39a1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.447844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-09d60512-1146-4366-87bb-60c71178ec4e None None] MFA Rules not processed for user `a270e9942e5840e094943db12eb10ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:53.449388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-09d60512-1146-4366-87bb-60c71178ec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:53.450223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3559a75fba2f40ccafb9a0f3b7bafbe1', 'request_id': 'req-723110e3-6c68-44da-95d6-147a18db39a1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.463069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] The token cache is being invalidate because project 3559a75fba2f40ccafb9a0f3b7bafbe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:53.463510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.464269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.466322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1236/2457] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:53 2022] DELETE /identity/v3/projects/3559a75fba2f40ccafb9a0f3b7bafbe1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:53.471171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:53.472859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:53.522445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1222/2458] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:07:53.534253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d60512-1146-4366-87bb-60c71178ec4e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:53.535639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-09d60512-1146-4366-87bb-60c71178ec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:53.544350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723110e3-6c68-44da-95d6-147a18db39a1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:53.550408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:53.551251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.551568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.551968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] PATH_INFO: `/v3/users/4effd461757841418964019a2bca194f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.553215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4effd461757841418964019a2bca194f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:53.555411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:53.601054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:53.605838 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d60512-1146-4366-87bb-60c71178ec4e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:53.610485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5758a59-895f-4603-af0e-b71a9687bc0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:53.611503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5758a59-895f-4603-af0e-b71a9687bc0f service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.611503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5758a59-895f-4603-af0e-b71a9687bc0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.611827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5758a59-895f-4603-af0e-b71a9687bc0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.616347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4effd461757841418964019a2bca194f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.625637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4effd461757841418964019a2bca194f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.626454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4effd461757841418964019a2bca194f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.627457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4effd461757841418964019a2bca194f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.628202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5758a59-895f-4603-af0e-b71a9687bc0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:53.629840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5758a59-895f-4603-af0e-b71a9687bc0f service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:53.644855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1223/2459] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:07:53.647532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-574f40f1-37ef-4799-8fb1-42ca03115b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4effd461757841418964019a2bca194f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.666628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1237/2460] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:53 2022] DELETE /identity/v3/users/4effd461757841418964019a2bca194f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:53.675671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5758a59-895f-4603-af0e-b71a9687bc0f service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:53.677912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c5758a59-895f-4603-af0e-b71a9687bc0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:53.691909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5758a59-895f-4603-af0e-b71a9687bc0f service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:53.776020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:53.777727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.778797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.779220 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] PATH_INFO: `/v3/users/a270e9942e5840e094943db12eb10ce2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.782707 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a270e9942e5840e094943db12eb10ce2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:53.785761 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:53.880678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a270e9942e5840e094943db12eb10ce2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.882607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a270e9942e5840e094943db12eb10ce2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.893740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a270e9942e5840e094943db12eb10ce2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.894638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a270e9942e5840e094943db12eb10ce2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.911639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a270e9942e5840e094943db12eb10ce2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:53.944244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1224/2461] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:07:53 2022] DELETE /identity/v3/users/a270e9942e5840e094943db12eb10ce2 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:53.949413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fea8f-3db3-4708-ab2a-cc98a001727e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:53.949971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fea8f-3db3-4708-ab2a-cc98a001727e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:53.950466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fea8f-3db3-4708-ab2a-cc98a001727e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:53.965855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:53.991273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c38fea8f-3db3-4708-ab2a-cc98a001727e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:53.995772 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c38fea8f-3db3-4708-ab2a-cc98a001727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:54.083260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1238/2462] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:54.094144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:54.095773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:54.120479 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fea8f-3db3-4708-ab2a-cc98a001727e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:54.122854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c38fea8f-3db3-4708-ab2a-cc98a001727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:54.153666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fea8f-3db3-4708-ab2a-cc98a001727e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:54.165768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:54.166884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:54.167663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:54.167956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] PATH_INFO: `/v3/projects/ce20193762824523845b1e32dee0e726` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:54.169467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce20193762824523845b1e32dee0e726)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:54.171516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:54.189009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c65915-3f6b-4872-a28b-98c8342492ae admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:54.202228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4aa0b9-417b-42d4-9a9d-e4eacf15753c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:54.203239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4aa0b9-417b-42d4-9a9d-e4eacf15753c service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:54.203239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4aa0b9-417b-42d4-9a9d-e4eacf15753c service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:54.203525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4aa0b9-417b-42d4-9a9d-e4eacf15753c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:54.214474 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4aa0b9-417b-42d4-9a9d-e4eacf15753c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:54.215950 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4aa0b9-417b-42d4-9a9d-e4eacf15753c service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:54.231405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1225/2463] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:54.239118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce20193762824523845b1e32dee0e726', 'request_id': 'req-d57f323f-fd72-44b4-ac10-ccb01d0683f4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:54.249837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce20193762824523845b1e32dee0e726', 'request_id': 'req-d57f323f-fd72-44b4-ac10-ccb01d0683f4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:54.259536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce20193762824523845b1e32dee0e726', 'request_id': 'req-d57f323f-fd72-44b4-ac10-ccb01d0683f4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:54.271055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] The token cache is being invalidate because project ce20193762824523845b1e32dee0e726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:54.271413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:54.272354 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d57f323f-fd72-44b4-ac10-ccb01d0683f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:54.274259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1239/2464] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:54 2022] DELETE /identity/v3/projects/ce20193762824523845b1e32dee0e726 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:54.545290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4aa0b9-417b-42d4-9a9d-e4eacf15753c service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:54.547422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ce4aa0b9-417b-42d4-9a9d-e4eacf15753c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:54.561298 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4aa0b9-417b-42d4-9a9d-e4eacf15753c service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:54.573916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:54.631898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:54.632778 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:54.633060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:54.633330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] PATH_INFO: `/v3/projects/61712d6374004516aaadcf07428b6c77` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:54.634539 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61712d6374004516aaadcf07428b6c77)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:54.636558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:54.689060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61712d6374004516aaadcf07428b6c77', 'request_id': 'req-b8a7a577-706b-482f-b1c8-8e95a0536d04'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:54.698892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61712d6374004516aaadcf07428b6c77', 'request_id': 'req-b8a7a577-706b-482f-b1c8-8e95a0536d04'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:54.706351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61712d6374004516aaadcf07428b6c77', 'request_id': 'req-b8a7a577-706b-482f-b1c8-8e95a0536d04'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:54.718606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] The token cache is being invalidate because project 61712d6374004516aaadcf07428b6c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:07:54.718863 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:54.719605 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b8a7a577-706b-482f-b1c8-8e95a0536d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:07:54.721588 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1226/2465] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:07:54 2022] DELETE /identity/v3/projects/61712d6374004516aaadcf07428b6c77 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:54.726825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7e9de-8661-413a-af7d-5d1a26c7a4e7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:54.727110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7e9de-8661-413a-af7d-5d1a26c7a4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:54.727422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7e9de-8661-413a-af7d-5d1a26c7a4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:54.752873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5fd7e9de-8661-413a-af7d-5d1a26c7a4e7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:54.754217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5fd7e9de-8661-413a-af7d-5d1a26c7a4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:54.822498 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1240/2466] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:54.830952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40842de0-ff01-41e2-b83d-48b6f403262f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:54.831215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40842de0-ff01-41e2-b83d-48b6f403262f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:54.831524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40842de0-ff01-41e2-b83d-48b6f403262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:54.851950 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-40842de0-ff01-41e2-b83d-48b6f403262f None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:54.853199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-40842de0-ff01-41e2-b83d-48b6f403262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:54.861807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1227/2467] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:54.866244 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7e9de-8661-413a-af7d-5d1a26c7a4e7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:54.876752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7e9de-8661-413a-af7d-5d1a26c7a4e7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:54.882322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f26bf-7091-4171-b03c-09f754539495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:54.882990 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f26bf-7091-4171-b03c-09f754539495 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:54.883274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f26bf-7091-4171-b03c-09f754539495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:54.883553 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f26bf-7091-4171-b03c-09f754539495 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:54.884306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f26bf-7091-4171-b03c-09f754539495 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:07:54.884711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f26bf-7091-4171-b03c-09f754539495 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:54.886390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f26bf-7091-4171-b03c-09f754539495 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:54.899078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1241/2468] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:07:54.903602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40842de0-ff01-41e2-b83d-48b6f403262f None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:54.914601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40842de0-ff01-41e2-b83d-48b6f403262f None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:54.919764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a290f-b529-419f-aac0-d71981c2d18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:54.920452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a290f-b529-419f-aac0-d71981c2d18d admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:54.920732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a290f-b529-419f-aac0-d71981c2d18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:54.921009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a290f-b529-419f-aac0-d71981c2d18d admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:54.921839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a290f-b529-419f-aac0-d71981c2d18d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:54.923832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a290f-b529-419f-aac0-d71981c2d18d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:54.952904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1228/2469] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:07:54 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:54.957209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f26bf-7091-4171-b03c-09f754539495 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:54.968160 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f26bf-7091-4171-b03c-09f754539495 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:54.981525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:55.033029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c123125-ef14-4538-b52e-9b64f74c0171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:55.033711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c123125-ef14-4538-b52e-9b64f74c0171 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:55.034001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c123125-ef14-4538-b52e-9b64f74c0171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:55.034332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c123125-ef14-4538-b52e-9b64f74c0171 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:55.035040 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c123125-ef14-4538-b52e-9b64f74c0171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:55.037036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c123125-ef14-4538-b52e-9b64f74c0171 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:55.058708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1242/2470] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:07:54 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:07:55.062837 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a290f-b529-419f-aac0-d71981c2d18d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:55.073933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a290f-b529-419f-aac0-d71981c2d18d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:55.079121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0610a-474a-4e03-a376-c3859641b479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:55.079901 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0610a-474a-4e03-a376-c3859641b479 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:55.080187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0610a-474a-4e03-a376-c3859641b479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:55.080463 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0610a-474a-4e03-a376-c3859641b479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:55.081204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0610a-474a-4e03-a376-c3859641b479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:55.082942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0610a-474a-4e03-a376-c3859641b479 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:55.094890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1229/2471] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:55.099016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c123125-ef14-4538-b52e-9b64f74c0171 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:55.109472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c123125-ef14-4538-b52e-9b64f74c0171 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:55.114682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c0163-1fc0-4391-9b7e-2cc61f17e2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:55.115332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c0163-1fc0-4391-9b7e-2cc61f17e2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:55.115613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c0163-1fc0-4391-9b7e-2cc61f17e2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:55.115889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c0163-1fc0-4391-9b7e-2cc61f17e2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:55.116622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c0163-1fc0-4391-9b7e-2cc61f17e2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:55.118280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c0163-1fc0-4391-9b7e-2cc61f17e2b5 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:55.128833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1243/2472] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:07:55.133121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0610a-474a-4e03-a376-c3859641b479 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:55.143586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0610a-474a-4e03-a376-c3859641b479 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:55.148587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83d71d-488a-43b2-beb1-e1e02639fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:55.149513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83d71d-488a-43b2-beb1-e1e02639fa6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:55.149797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83d71d-488a-43b2-beb1-e1e02639fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:55.150077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83d71d-488a-43b2-beb1-e1e02639fa6a admin admin] PATH_INFO: `/v3/projects/7a4f3c7eab96453cb7e06adbb72b5e16/users/ae2206529b5d4e75ac948f9669a46ea7/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:55.165408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83d71d-488a-43b2-beb1-e1e02639fa6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a4f3c7eab96453cb7e06adbb72b5e16, user_id=ae2206529b5d4e75ac948f9669a46ea7, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:55.168120 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83d71d-488a-43b2-beb1-e1e02639fa6a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:55.182659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1230/2473] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:07:55 2022] PUT /identity/v3/projects/7a4f3c7eab96453cb7e06adbb72b5e16/users/ae2206529b5d4e75ac948f9669a46ea7/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:07:55.193659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b490e6-2170-4781-af38-d92c2257b7e5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:55.193939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b490e6-2170-4781-af38-d92c2257b7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:55.194248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b490e6-2170-4781-af38-d92c2257b7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:55.230153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-41b490e6-2170-4781-af38-d92c2257b7e5 None None] MFA Rules not processed for user `ae2206529b5d4e75ac948f9669a46ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:07:55.231893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-41b490e6-2170-4781-af38-d92c2257b7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:55.356964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1244/2474] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:07:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:07:55.369555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83d71d-488a-43b2-beb1-e1e02639fa6a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:55.372084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0c83d71d-488a-43b2-beb1-e1e02639fa6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:55.434039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:55.450735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83d71d-488a-43b2-beb1-e1e02639fa6a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:55.455010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee860d45-e4c7-45a3-b24c-94d9df1d67a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:55.455698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee860d45-e4c7-45a3-b24c-94d9df1d67a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:55.456042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee860d45-e4c7-45a3-b24c-94d9df1d67a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:55.456323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee860d45-e4c7-45a3-b24c-94d9df1d67a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:55.471913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee860d45-e4c7-45a3-b24c-94d9df1d67a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:55.473390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee860d45-e4c7-45a3-b24c-94d9df1d67a6 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:55.484410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1231/2475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:07:56.535641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b490e6-2170-4781-af38-d92c2257b7e5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:07:56.537316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-41b490e6-2170-4781-af38-d92c2257b7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:56.600029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b490e6-2170-4781-af38-d92c2257b7e5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:07:56.603695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d1865-a2ad-4125-bebc-78edf471ddf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:07:56.604519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d1865-a2ad-4125-bebc-78edf471ddf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:07:56.604804 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d1865-a2ad-4125-bebc-78edf471ddf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:07:56.605087 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d1865-a2ad-4125-bebc-78edf471ddf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:07:56.606928 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5b3d1865-a2ad-4125-bebc-78edf471ddf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:07:56.651853 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:07:56.682667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d1865-a2ad-4125-bebc-78edf471ddf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:07:56.684159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d1865-a2ad-4125-bebc-78edf471ddf1 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:07:56.694592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1245/2476] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:07:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:08:05.480349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=8 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:08:06.689702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=11 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:08:35.409855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3057b9-a555-4908-8f47-976cd7e64f3f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:35.410575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3057b9-a555-4908-8f47-976cd7e64f3f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:35.410575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3057b9-a555-4908-8f47-976cd7e64f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:35.429389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-5e3057b9-a555-4908-8f47-976cd7e64f3f None None] MFA Rules not processed for user `a96fa98fa00c4de29a9bed22ed9e597b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:08:35.431340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5e3057b9-a555-4908-8f47-976cd7e64f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:35.490794 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1232/2477] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:08:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:08:37.659674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d1865-a2ad-4125-bebc-78edf471ddf1 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:37.740954 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d1865-a2ad-4125-bebc-78edf471ddf1 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:37.746206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:37.747354 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:37.747757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:37.748132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] PATH_INFO: `/v3/users/ae2206529b5d4e75ac948f9669a46ea7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:37.749501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae2206529b5d4e75ac948f9669a46ea7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:37.751578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:37.810290 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae2206529b5d4e75ac948f9669a46ea7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:37.820222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae2206529b5d4e75ac948f9669a46ea7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:37.821452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae2206529b5d4e75ac948f9669a46ea7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:37.822122 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae2206529b5d4e75ac948f9669a46ea7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:37.838128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae2206529b5d4e75ac948f9669a46ea7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:37.853903 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1246/2478] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:08:37 2022] DELETE /identity/v3/users/ae2206529b5d4e75ac948f9669a46ea7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:08:37.858417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e9a6-29da-4a4a-9cdf-a09d462054fb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:37.858917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e9a6-29da-4a4a-9cdf-a09d462054fb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:37.859033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4e9a6-29da-4a4a-9cdf-a09d462054fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:37.877859 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6bc4e9a6-29da-4a4a-9cdf-a09d462054fb None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:08:37.879145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6bc4e9a6-29da-4a4a-9cdf-a09d462054fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:37.966180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1233/2479] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:08:37.974459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:37.976487 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:38.055966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7c8a5-abe7-457c-ae02-7d8c9766d647 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:38.059981 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a4344-118a-4a11-ba95-d150a60d77db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:38.060718 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a4344-118a-4a11-ba95-d150a60d77db service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:38.061202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a4344-118a-4a11-ba95-d150a60d77db service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:38.061577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a4344-118a-4a11-ba95-d150a60d77db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:38.072427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a4344-118a-4a11-ba95-d150a60d77db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:38.073889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a4344-118a-4a11-ba95-d150a60d77db service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:38.084175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1247/2480] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:08:38.587481 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e9a6-29da-4a4a-9cdf-a09d462054fb None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:38.589203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6bc4e9a6-29da-4a4a-9cdf-a09d462054fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:38.603929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4e9a6-29da-4a4a-9cdf-a09d462054fb None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:38.609312 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:38.610403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:38.610754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:38.611040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] PATH_INFO: `/v3/projects/7a4f3c7eab96453cb7e06adbb72b5e16` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:38.612460 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a4f3c7eab96453cb7e06adbb72b5e16)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:38.619870 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:38.681082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a4f3c7eab96453cb7e06adbb72b5e16', 'request_id': 'req-2a4b3561-e80a-487b-b9c1-7a419d74936b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:38.693839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a4f3c7eab96453cb7e06adbb72b5e16', 'request_id': 'req-2a4b3561-e80a-487b-b9c1-7a419d74936b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:38.706226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a4f3c7eab96453cb7e06adbb72b5e16', 'request_id': 'req-2a4b3561-e80a-487b-b9c1-7a419d74936b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:38.721798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] The token cache is being invalidate because project 7a4f3c7eab96453cb7e06adbb72b5e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:08:38.722027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:38.722692 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a4b3561-e80a-487b-b9c1-7a419d74936b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:38.724844 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1234/2481] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:08:38 2022] DELETE /identity/v3/projects/7a4f3c7eab96453cb7e06adbb72b5e16 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:08:38.729704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4d30-d4e7-4a44-a1cc-c8a32b39e24c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:38.730276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4d30-d4e7-4a44-a1cc-c8a32b39e24c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:38.730609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4d30-d4e7-4a44-a1cc-c8a32b39e24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:38.763373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6edf4d30-d4e7-4a44-a1cc-c8a32b39e24c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:08:38.765507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6edf4d30-d4e7-4a44-a1cc-c8a32b39e24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:38.786936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:08:38.848681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1248/2482] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:08:38.856059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0e790-8381-4090-b857-8dddfc5254d6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:38.857091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0e790-8381-4090-b857-8dddfc5254d6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:38.857522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0e790-8381-4090-b857-8dddfc5254d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:38.880246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8be0e790-8381-4090-b857-8dddfc5254d6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:08:38.882618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8be0e790-8381-4090-b857-8dddfc5254d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:38.891583 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1235/2483] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:08:38.896241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4d30-d4e7-4a44-a1cc-c8a32b39e24c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:38.910567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4d30-d4e7-4a44-a1cc-c8a32b39e24c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:38.917667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06618bd5-baa2-4f07-9e0a-3a11e8d8df3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:38.918464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06618bd5-baa2-4f07-9e0a-3a11e8d8df3c admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:38.919022 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06618bd5-baa2-4f07-9e0a-3a11e8d8df3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:38.919403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06618bd5-baa2-4f07-9e0a-3a11e8d8df3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:38.920206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06618bd5-baa2-4f07-9e0a-3a11e8d8df3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:08:38.920704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06618bd5-baa2-4f07-9e0a-3a11e8d8df3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:38.922611 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06618bd5-baa2-4f07-9e0a-3a11e8d8df3c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:38.958065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1249/2484] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:08:38.962032 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0e790-8381-4090-b857-8dddfc5254d6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:38.972650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0e790-8381-4090-b857-8dddfc5254d6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:38.986937 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4eea09-47f3-4da4-874b-26e3f08d0989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:38.987627 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4eea09-47f3-4da4-874b-26e3f08d0989 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:38.988074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4eea09-47f3-4da4-874b-26e3f08d0989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:38.988353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4eea09-47f3-4da4-874b-26e3f08d0989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:38.989123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4eea09-47f3-4da4-874b-26e3f08d0989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:38.991190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4eea09-47f3-4da4-874b-26e3f08d0989 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:39.015542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:08:39.025186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1236/2485] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:08:38 2022] POST /identity/v3/projects => generated 397 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:08:39.029151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06618bd5-baa2-4f07-9e0a-3a11e8d8df3c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:39.040084 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06618bd5-baa2-4f07-9e0a-3a11e8d8df3c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:39.100801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49279a-488e-4516-aad7-9728ac079865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:39.101512 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49279a-488e-4516-aad7-9728ac079865 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:39.101878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49279a-488e-4516-aad7-9728ac079865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:39.102778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49279a-488e-4516-aad7-9728ac079865 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:39.103481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49279a-488e-4516-aad7-9728ac079865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:39.105641 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49279a-488e-4516-aad7-9728ac079865 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:39.127918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1250/2486] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:08:39 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:08:39.132152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4eea09-47f3-4da4-874b-26e3f08d0989 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:39.144341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4eea09-47f3-4da4-874b-26e3f08d0989 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:39.149842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b88b91-9b35-4cfc-97d2-b30d101ea597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:39.150570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b88b91-9b35-4cfc-97d2-b30d101ea597 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:39.150850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b88b91-9b35-4cfc-97d2-b30d101ea597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:39.151124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b88b91-9b35-4cfc-97d2-b30d101ea597 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:39.151861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b88b91-9b35-4cfc-97d2-b30d101ea597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:39.153487 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b88b91-9b35-4cfc-97d2-b30d101ea597 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:39.165441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1237/2487] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:08:39.169530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49279a-488e-4516-aad7-9728ac079865 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:39.182572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49279a-488e-4516-aad7-9728ac079865 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:39.189890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddad7a3-30d8-4d39-ac89-e1fe48b2a367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:39.190617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddad7a3-30d8-4d39-ac89-e1fe48b2a367 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:39.190980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddad7a3-30d8-4d39-ac89-e1fe48b2a367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:39.191625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddad7a3-30d8-4d39-ac89-e1fe48b2a367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:39.192497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddad7a3-30d8-4d39-ac89-e1fe48b2a367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:39.194203 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddad7a3-30d8-4d39-ac89-e1fe48b2a367 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:39.209014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1251/2488] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:08:39.210272 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b88b91-9b35-4cfc-97d2-b30d101ea597 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:39.221757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b88b91-9b35-4cfc-97d2-b30d101ea597 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:39.227718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063bf8f-c6de-4c76-b12c-edee6233762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:39.228645 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063bf8f-c6de-4c76-b12c-edee6233762a admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:39.228986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063bf8f-c6de-4c76-b12c-edee6233762a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:39.229280 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063bf8f-c6de-4c76-b12c-edee6233762a admin admin] PATH_INFO: `/v3/projects/81a6ad46af904f6da267853a3e404e59/users/90f2477dd03e499f8b437986dd6ead8c/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:39.247468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063bf8f-c6de-4c76-b12c-edee6233762a admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a6ad46af904f6da267853a3e404e59, user_id=90f2477dd03e499f8b437986dd6ead8c, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:39.250379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063bf8f-c6de-4c76-b12c-edee6233762a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:39.266162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1238/2489] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:08:39 2022] PUT /identity/v3/projects/81a6ad46af904f6da267853a3e404e59/users/90f2477dd03e499f8b437986dd6ead8c/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:08:39.277066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67867b-77c6-4e48-9e57-be6e9e4358f5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:39.277525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67867b-77c6-4e48-9e57-be6e9e4358f5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:39.277803 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67867b-77c6-4e48-9e57-be6e9e4358f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:39.310293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ac67867b-77c6-4e48-9e57-be6e9e4358f5 None None] MFA Rules not processed for user `90f2477dd03e499f8b437986dd6ead8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:08:39.311488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ac67867b-77c6-4e48-9e57-be6e9e4358f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:39.392988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1252/2490] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:08:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:08:39.407419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063bf8f-c6de-4c76-b12c-edee6233762a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:39.409550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d063bf8f-c6de-4c76-b12c-edee6233762a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:39.479684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063bf8f-c6de-4c76-b12c-edee6233762a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:39.483513 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e43590-7869-4b50-ba9d-b4965e3f0c27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:39.484332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e43590-7869-4b50-ba9d-b4965e3f0c27 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:39.484735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e43590-7869-4b50-ba9d-b4965e3f0c27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:39.485116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e43590-7869-4b50-ba9d-b4965e3f0c27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:39.495422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e43590-7869-4b50-ba9d-b4965e3f0c27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:39.497447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e43590-7869-4b50-ba9d-b4965e3f0c27 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:39.509705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1239/2491] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:08:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:08:49.390540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:08:49.504910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:08:51.921074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67867b-77c6-4e48-9e57-be6e9e4358f5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:51.995686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67867b-77c6-4e48-9e57-be6e9e4358f5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:52.001158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:52.002139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:52.002494 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:52.002825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] PATH_INFO: `/v3/users/90f2477dd03e499f8b437986dd6ead8c` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:52.004243 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f2477dd03e499f8b437986dd6ead8c)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:52.006469 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:52.062685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f2477dd03e499f8b437986dd6ead8c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:52.073240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f2477dd03e499f8b437986dd6ead8c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:52.073894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f2477dd03e499f8b437986dd6ead8c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:52.074657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f2477dd03e499f8b437986dd6ead8c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:52.088304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f2477dd03e499f8b437986dd6ead8c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:52.104136 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1253/2492] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:08:51 2022] DELETE /identity/v3/users/90f2477dd03e499f8b437986dd6ead8c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:08:52.109022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11b9e7-aa5a-4a91-a60a-73e8cbb5b128 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:52.109506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11b9e7-aa5a-4a91-a60a-73e8cbb5b128 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:52.109597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11b9e7-aa5a-4a91-a60a-73e8cbb5b128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:52.129191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-fd11b9e7-aa5a-4a91-a60a-73e8cbb5b128 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:08:52.130485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fd11b9e7-aa5a-4a91-a60a-73e8cbb5b128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:52.195747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1240/2493] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:08:52.206193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:52.207840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:52.284390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed9635-b695-479b-a2d7-346b8135e3a7 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:52.288673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9059f-e6f2-4ed4-a7dc-a09c6d49f0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:52.289360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9059f-e6f2-4ed4-a7dc-a09c6d49f0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:52.289631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9059f-e6f2-4ed4-a7dc-a09c6d49f0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:52.289904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9059f-e6f2-4ed4-a7dc-a09c6d49f0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:52.300525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9059f-e6f2-4ed4-a7dc-a09c6d49f0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:52.302140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9059f-e6f2-4ed4-a7dc-a09c6d49f0c0 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:52.313789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1254/2494] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:08:52.606282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11b9e7-aa5a-4a91-a60a-73e8cbb5b128 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:08:52.607718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fd11b9e7-aa5a-4a91-a60a-73e8cbb5b128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:08:52.622579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11b9e7-aa5a-4a91-a60a-73e8cbb5b128 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:08:52.630001 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:08:52.630951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:08:52.631610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:08:52.631610 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] PATH_INFO: `/v3/projects/81a6ad46af904f6da267853a3e404e59` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:08:52.633660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a6ad46af904f6da267853a3e404e59)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:08:52.635571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:08:52.692388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a6ad46af904f6da267853a3e404e59', 'request_id': 'req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:52.701016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a6ad46af904f6da267853a3e404e59', 'request_id': 'req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:52.708040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a6ad46af904f6da267853a3e404e59', 'request_id': 'req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:52.721710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] The token cache is being invalidate because project 81a6ad46af904f6da267853a3e404e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:08:52.721996 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:52.722952 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:08:52.724691 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1241/2495] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:08:52 2022] DELETE /identity/v3/projects/81a6ad46af904f6da267853a3e404e59 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:09:02.308778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:09:02.718555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:09:04.474274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9059f-e6f2-4ed4-a7dc-a09c6d49f0c0 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:04.475999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8bc9059f-e6f2-4ed4-a7dc-a09c6d49f0c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:04.490256 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9059f-e6f2-4ed4-a7dc-a09c6d49f0c0 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:04.546614 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb23887-f308-4b7d-8bc4-815d6ad6c256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:04.547277 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb23887-f308-4b7d-8bc4-815d6ad6c256 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:04.547585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb23887-f308-4b7d-8bc4-815d6ad6c256 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:04.547914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb23887-f308-4b7d-8bc4-815d6ad6c256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:04.550121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dcb23887-f308-4b7d-8bc4-815d6ad6c256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:04.620046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb23887-f308-4b7d-8bc4-815d6ad6c256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:04.621636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb23887-f308-4b7d-8bc4-815d6ad6c256 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:04.633286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1255/2496] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:09:04 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:09:14.628378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=50 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:09:20.416556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:20.418363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:20.433426 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d39c6-b563-4bc0-bec6-bcc6dc5fa955 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:20.437846 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13401d8-c7bc-4994-ba3d-1b00a8a8442c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:20.439135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13401d8-c7bc-4994-ba3d-1b00a8a8442c service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:20.439135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13401d8-c7bc-4994-ba3d-1b00a8a8442c service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:20.439507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13401d8-c7bc-4994-ba3d-1b00a8a8442c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:20.443207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d13401d8-c7bc-4994-ba3d-1b00a8a8442c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:20.549985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13401d8-c7bc-4994-ba3d-1b00a8a8442c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:20.551741 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13401d8-c7bc-4994-ba3d-1b00a8a8442c service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:20.565109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1242/2497] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:09:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:09:22.142415 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb23887-f308-4b7d-8bc4-815d6ad6c256 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:22.144026 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dcb23887-f308-4b7d-8bc4-815d6ad6c256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:22.155897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb23887-f308-4b7d-8bc4-815d6ad6c256 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:22.160851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25529234-10c8-44c0-90aa-614d3b5e7702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:22.161489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25529234-10c8-44c0-90aa-614d3b5e7702 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:22.161769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25529234-10c8-44c0-90aa-614d3b5e7702 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:22.162116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25529234-10c8-44c0-90aa-614d3b5e7702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:22.163980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-25529234-10c8-44c0-90aa-614d3b5e7702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:22.180457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25529234-10c8-44c0-90aa-614d3b5e7702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:22.181930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25529234-10c8-44c0-90aa-614d3b5e7702 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:22.196606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1256/2498] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:09:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:09:30.559366 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=34 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:09:32.189460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=6 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:09:57.156510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13401d8-c7bc-4994-ba3d-1b00a8a8442c service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:57.158586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d13401d8-c7bc-4994-ba3d-1b00a8a8442c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:57.234980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25529234-10c8-44c0-90aa-614d3b5e7702 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:57.237642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-25529234-10c8-44c0-90aa-614d3b5e7702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:57.250176 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13401d8-c7bc-4994-ba3d-1b00a8a8442c service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:57.258206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aceff6-efca-4a36-957a-415b52825157 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:57.259283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aceff6-efca-4a36-957a-415b52825157 service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:57.259720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aceff6-efca-4a36-957a-415b52825157 service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:57.260139 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aceff6-efca-4a36-957a-415b52825157 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:57.263936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-66aceff6-efca-4a36-957a-415b52825157 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:57.283460 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aceff6-efca-4a36-957a-415b52825157 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:57.286554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aceff6-efca-4a36-957a-415b52825157 service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:57.300934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1243/2499] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:09:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:09:57.325581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25529234-10c8-44c0-90aa-614d3b5e7702 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:57.337854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:57.338794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:57.339073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:57.339346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] PATH_INFO: `/v3/users/a96fa98fa00c4de29a9bed22ed9e597b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:57.340554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a96fa98fa00c4de29a9bed22ed9e597b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:57.342676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:57.406431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a96fa98fa00c4de29a9bed22ed9e597b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:09:57.417796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a96fa98fa00c4de29a9bed22ed9e597b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:09:57.423197 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a96fa98fa00c4de29a9bed22ed9e597b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:09:57.424202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a96fa98fa00c4de29a9bed22ed9e597b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:09:57.437571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0a8128d8-ef9e-4cee-b32a-cd42cf33d931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a96fa98fa00c4de29a9bed22ed9e597b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:09:57.453608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1257/2500] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:09:57 2022] DELETE /identity/v3/users/a96fa98fa00c4de29a9bed22ed9e597b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:09:57.660708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aceff6-efca-4a36-957a-415b52825157 service placement] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:57.671802 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aceff6-efca-4a36-957a-415b52825157 service placement] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:57.740153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:57.741822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:57.741822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:57.742031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] PATH_INFO: `/v3/projects/d47c143cf33a41799202d5f7d90a66b5` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:57.743258 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d47c143cf33a41799202d5f7d90a66b5)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:57.745215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:57.796309 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd47c143cf33a41799202d5f7d90a66b5', 'request_id': 'req-ee93ef88-f1df-4300-81be-20452556a537'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:09:57.808274 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd47c143cf33a41799202d5f7d90a66b5', 'request_id': 'req-ee93ef88-f1df-4300-81be-20452556a537'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:09:57.815906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd47c143cf33a41799202d5f7d90a66b5', 'request_id': 'req-ee93ef88-f1df-4300-81be-20452556a537'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:09:57.829177 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] The token cache is being invalidate because project d47c143cf33a41799202d5f7d90a66b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:09:57.829467 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:09:57.830238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ee93ef88-f1df-4300-81be-20452556a537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:09:57.831888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1244/2501] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:09:57 2022] DELETE /identity/v3/projects/d47c143cf33a41799202d5f7d90a66b5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:09:57.837561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ea1c-824c-4957-8b53-3e975da61d26 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:57.837837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ea1c-824c-4957-8b53-3e975da61d26 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:57.838118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015ea1c-824c-4957-8b53-3e975da61d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:57.857754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b015ea1c-824c-4957-8b53-3e975da61d26 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:09:57.858991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b015ea1c-824c-4957-8b53-3e975da61d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:57.930214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1258/2502] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:09:57.938417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731c221-356e-4707-be99-57a3ee5b6edc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:57.938718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731c221-356e-4707-be99-57a3ee5b6edc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:57.939024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731c221-356e-4707-be99-57a3ee5b6edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:57.959779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f731c221-356e-4707-be99-57a3ee5b6edc None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:09:57.962234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f731c221-356e-4707-be99-57a3ee5b6edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:57.970234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1245/2503] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:09:57.974842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015ea1c-824c-4957-8b53-3e975da61d26 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:57.985267 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015ea1c-824c-4957-8b53-3e975da61d26 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:57.990161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe94be7-5cee-4d93-8a49-9ec44c8d5fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:57.990892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe94be7-5cee-4d93-8a49-9ec44c8d5fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:57.991537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe94be7-5cee-4d93-8a49-9ec44c8d5fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:57.991537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe94be7-5cee-4d93-8a49-9ec44c8d5fb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:57.992186 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe94be7-5cee-4d93-8a49-9ec44c8d5fb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:09:57.992499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe94be7-5cee-4d93-8a49-9ec44c8d5fb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:57.994230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe94be7-5cee-4d93-8a49-9ec44c8d5fb1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:58.006118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1259/2504] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:09:58.010890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731c221-356e-4707-be99-57a3ee5b6edc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:58.021246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731c221-356e-4707-be99-57a3ee5b6edc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:58.026687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dd297-bb2b-4929-9018-66c2ab1d9b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:58.027402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dd297-bb2b-4929-9018-66c2ab1d9b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:58.027747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dd297-bb2b-4929-9018-66c2ab1d9b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:58.027951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dd297-bb2b-4929-9018-66c2ab1d9b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:58.028648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dd297-bb2b-4929-9018-66c2ab1d9b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:58.030629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dd297-bb2b-4929-9018-66c2ab1d9b84 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:58.041651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:09:58.069759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1246/2505] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:09:58 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:09:58.071521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe94be7-5cee-4d93-8a49-9ec44c8d5fb1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:58.085137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe94be7-5cee-4d93-8a49-9ec44c8d5fb1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:58.092719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:09:58.160030 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cb12f-1373-4305-a336-4566e6f462b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:58.160618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cb12f-1373-4305-a336-4566e6f462b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:58.160906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cb12f-1373-4305-a336-4566e6f462b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:58.161171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cb12f-1373-4305-a336-4566e6f462b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:58.161901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cb12f-1373-4305-a336-4566e6f462b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:58.164356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cb12f-1373-4305-a336-4566e6f462b0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:58.187184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1260/2506] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:09:58 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:09:58.192081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dd297-bb2b-4929-9018-66c2ab1d9b84 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:58.203634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dd297-bb2b-4929-9018-66c2ab1d9b84 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:58.211267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1d106-e27f-4b64-a730-37af444ecc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:58.212088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1d106-e27f-4b64-a730-37af444ecc57 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:58.212527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1d106-e27f-4b64-a730-37af444ecc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:58.212978 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1d106-e27f-4b64-a730-37af444ecc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:58.213762 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1d106-e27f-4b64-a730-37af444ecc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:58.215437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1d106-e27f-4b64-a730-37af444ecc57 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:58.230857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1247/2507] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:09:58.233778 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cb12f-1373-4305-a336-4566e6f462b0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:58.245795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cb12f-1373-4305-a336-4566e6f462b0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:58.251476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d99e32-8985-467b-a213-846951624de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:58.252172 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d99e32-8985-467b-a213-846951624de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:58.252452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d99e32-8985-467b-a213-846951624de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:58.252726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d99e32-8985-467b-a213-846951624de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:58.253488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d99e32-8985-467b-a213-846951624de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:58.255189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d99e32-8985-467b-a213-846951624de4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:58.267771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1261/2508] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:09:58.272456 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1d106-e27f-4b64-a730-37af444ecc57 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:58.285399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1d106-e27f-4b64-a730-37af444ecc57 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:58.292278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ec5ee-5798-48d6-abd9-ab5225bd0353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:58.293238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ec5ee-5798-48d6-abd9-ab5225bd0353 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:58.293521 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ec5ee-5798-48d6-abd9-ab5225bd0353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:58.293798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ec5ee-5798-48d6-abd9-ab5225bd0353 admin admin] PATH_INFO: `/v3/projects/fdaa800381f948619a950d65f637c16a/users/ca6820ed031b49a8b3f6cff4ad7a899d/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:58.310048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ec5ee-5798-48d6-abd9-ab5225bd0353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdaa800381f948619a950d65f637c16a, user_id=ca6820ed031b49a8b3f6cff4ad7a899d, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:58.312785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ec5ee-5798-48d6-abd9-ab5225bd0353 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:58.328185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1248/2509] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:09:58 2022] PUT /identity/v3/projects/fdaa800381f948619a950d65f637c16a/users/ca6820ed031b49a8b3f6cff4ad7a899d/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:09:58.335831 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded920f3-9b63-40d6-ba03-2637891413d3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:58.336116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded920f3-9b63-40d6-ba03-2637891413d3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:58.336395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded920f3-9b63-40d6-ba03-2637891413d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:58.360148 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ded920f3-9b63-40d6-ba03-2637891413d3 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:09:58.361388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ded920f3-9b63-40d6-ba03-2637891413d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:58.429329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1262/2510] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:09:58.441964 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ec5ee-5798-48d6-abd9-ab5225bd0353 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:09:58.444057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-813ec5ee-5798-48d6-abd9-ab5225bd0353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:09:58.510047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ec5ee-5798-48d6-abd9-ab5225bd0353 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:09:58.514052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f253e6e-da58-4c4d-ad94-e40a8a658a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:09:58.514733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f253e6e-da58-4c4d-ad94-e40a8a658a9b service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:09:58.515012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f253e6e-da58-4c4d-ad94-e40a8a658a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:09:58.515284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f253e6e-da58-4c4d-ad94-e40a8a658a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:09:58.529721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f253e6e-da58-4c4d-ad94-e40a8a658a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:09:58.531254 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f253e6e-da58-4c4d-ad94-e40a8a658a9b service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:09:58.541649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1249/2511] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:10:02.911890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d3448-968b-4f67-93e3-5f59d426ca6e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:02.912409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d3448-968b-4f67-93e3-5f59d426ca6e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:02.912588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d3448-968b-4f67-93e3-5f59d426ca6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:02.943260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4c5d3448-968b-4f67-93e3-5f59d426ca6e None None] MFA Rules not processed for user `ca6820ed031b49a8b3f6cff4ad7a899d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:10:02.944565 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4c5d3448-968b-4f67-93e3-5f59d426ca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:03.030142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1263/2512] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:10:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:10:03.035006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f253e6e-da58-4c4d-ad94-e40a8a658a9b service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:03.049249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f253e6e-da58-4c4d-ad94-e40a8a658a9b service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:03.055503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25079a-e6a9-4278-8a09-f15fc68894f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:03.056286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25079a-e6a9-4278-8a09-f15fc68894f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:03.056630 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25079a-e6a9-4278-8a09-f15fc68894f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:03.056966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25079a-e6a9-4278-8a09-f15fc68894f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:03.057739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25079a-e6a9-4278-8a09-f15fc68894f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:03.059765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25079a-e6a9-4278-8a09-f15fc68894f6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:03.092149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1250/2513] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:10:03 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:10:03.096418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d3448-968b-4f67-93e3-5f59d426ca6e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:03.108472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d3448-968b-4f67-93e3-5f59d426ca6e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:03.121973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:10:03.174969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fba2cb-f145-43a2-a075-2ea73539f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:03.175659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fba2cb-f145-43a2-a075-2ea73539f7dc admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:03.175943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fba2cb-f145-43a2-a075-2ea73539f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:03.176220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fba2cb-f145-43a2-a075-2ea73539f7dc admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:03.176929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fba2cb-f145-43a2-a075-2ea73539f7dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:03.179231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fba2cb-f145-43a2-a075-2ea73539f7dc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:03.201232 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1264/2514] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:10:03 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:10:03.205264 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25079a-e6a9-4278-8a09-f15fc68894f6 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:03.217205 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25079a-e6a9-4278-8a09-f15fc68894f6 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:03.222849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3ba3d-4e95-4f84-8616-24f36bf53cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:03.223732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ba3d-4e95-4f84-8616-24f36bf53cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:03.224136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ba3d-4e95-4f84-8616-24f36bf53cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:03.224462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3ba3d-4e95-4f84-8616-24f36bf53cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:03.225546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3ba3d-4e95-4f84-8616-24f36bf53cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:03.227695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3ba3d-4e95-4f84-8616-24f36bf53cdf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:03.241846 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1251/2515] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:10:03.245925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fba2cb-f145-43a2-a075-2ea73539f7dc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:03.259397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fba2cb-f145-43a2-a075-2ea73539f7dc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:03.265700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d29958-038c-485c-a6ee-4942eff5284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:03.267083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d29958-038c-485c-a6ee-4942eff5284a admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:03.267511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d29958-038c-485c-a6ee-4942eff5284a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:03.267933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d29958-038c-485c-a6ee-4942eff5284a admin admin] PATH_INFO: `/v3/projects/1d2977bee89c4b65938cfb896c02fbee/users/3ef07f57ca4041baae444ec89958c62f/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:03.292294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d29958-038c-485c-a6ee-4942eff5284a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2977bee89c4b65938cfb896c02fbee, user_id=3ef07f57ca4041baae444ec89958c62f, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:03.295036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d29958-038c-485c-a6ee-4942eff5284a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:03.318118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1265/2516] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:10:03 2022] PUT /identity/v3/projects/1d2977bee89c4b65938cfb896c02fbee/users/3ef07f57ca4041baae444ec89958c62f/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:10:03.324451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3ba3d-4e95-4f84-8616-24f36bf53cdf admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:03.337310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3ba3d-4e95-4f84-8616-24f36bf53cdf admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:03.408164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c55a5-388d-4a7d-a426-b3ed3ec8a7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:03.408857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c55a5-388d-4a7d-a426-b3ed3ec8a7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:03.409150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c55a5-388d-4a7d-a426-b3ed3ec8a7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:03.409419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c55a5-388d-4a7d-a426-b3ed3ec8a7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:03.410218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c55a5-388d-4a7d-a426-b3ed3ec8a7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:03.411915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c55a5-388d-4a7d-a426-b3ed3ec8a7fe admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:03.426145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1252/2517] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:10:03.429361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d29958-038c-485c-a6ee-4942eff5284a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:03.446546 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d29958-038c-485c-a6ee-4942eff5284a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:03.455463 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb0886-a31a-40cf-9b83-96b127756470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:03.456459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb0886-a31a-40cf-9b83-96b127756470 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:03.456674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb0886-a31a-40cf-9b83-96b127756470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:03.456949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb0886-a31a-40cf-9b83-96b127756470 admin admin] PATH_INFO: `/v3/projects/1d2977bee89c4b65938cfb896c02fbee/users/3ef07f57ca4041baae444ec89958c62f/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:03.459294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb0886-a31a-40cf-9b83-96b127756470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2977bee89c4b65938cfb896c02fbee, user_id=3ef07f57ca4041baae444ec89958c62f, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:03.474991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb0886-a31a-40cf-9b83-96b127756470 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:03.495337 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1266/2518] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:10:03 2022] PUT /identity/v3/projects/1d2977bee89c4b65938cfb896c02fbee/users/3ef07f57ca4041baae444ec89958c62f/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:10:08.290776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66cc04-df35-4744-82bd-1c2729e62813 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:08.291936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66cc04-df35-4744-82bd-1c2729e62813 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:08.291936 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66cc04-df35-4744-82bd-1c2729e62813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:08.333155 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-df66cc04-df35-4744-82bd-1c2729e62813 None None] MFA Rules not processed for user `3ef07f57ca4041baae444ec89958c62f`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:10:08.335340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-df66cc04-df35-4744-82bd-1c2729e62813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:08.353306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:10:08.460931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1253/2519] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:10:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:10:08.484322 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb0886-a31a-40cf-9b83-96b127756470 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:08.487630 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1dbb0886-a31a-40cf-9b83-96b127756470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:08.648511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb0886-a31a-40cf-9b83-96b127756470 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:08.655766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df413b-4e84-4f36-8c0c-7bc9fc888184 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bf4d9571d541bcbba87094198dc2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:08.656893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df413b-4e84-4f36-8c0c-7bc9fc888184 service glance] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:08.657373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df413b-4e84-4f36-8c0c-7bc9fc888184 service glance] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:08.657859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df413b-4e84-4f36-8c0c-7bc9fc888184 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:08.743918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df413b-4e84-4f36-8c0c-7bc9fc888184 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:08.745645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df413b-4e84-4f36-8c0c-7bc9fc888184 service glance] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:08.755842 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1267/2520] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:10:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:10:08.942270 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188fff4-4959-45a9-9cce-6b15161ccbc9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:08.942562 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188fff4-4959-45a9-9cce-6b15161ccbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:08.942843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188fff4-4959-45a9-9cce-6b15161ccbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:09.006480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-6188fff4-4959-45a9-9cce-6b15161ccbc9 None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:10:09.007828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6188fff4-4959-45a9-9cce-6b15161ccbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:09.099916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1254/2521] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:10:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:10:09.162032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df413b-4e84-4f36-8c0c-7bc9fc888184 service glance] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:09.166574 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-14df413b-4e84-4f36-8c0c-7bc9fc888184 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:09.170833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:10:09.286424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df413b-4e84-4f36-8c0c-7bc9fc888184 service glance] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:09.293296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed22e3-6b4e-4d29-983f-1bb7672da154 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:09.293972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed22e3-6b4e-4d29-983f-1bb7672da154 service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:09.294288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed22e3-6b4e-4d29-983f-1bb7672da154 service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:09.294561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed22e3-6b4e-4d29-983f-1bb7672da154 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:09.304130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed22e3-6b4e-4d29-983f-1bb7672da154 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:09.305918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed22e3-6b4e-4d29-983f-1bb7672da154 service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:09.316146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1268/2522] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:10:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 20 11:10:18.263534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba615bc4-1c05-4f0a-a01f-b7e13a63a419 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:18.264104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba615bc4-1c05-4f0a-a01f-b7e13a63a419 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:18.264104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba615bc4-1c05-4f0a-a01f-b7e13a63a419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:18.282722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ba615bc4-1c05-4f0a-a01f-b7e13a63a419 None None] MFA Rules not processed for user `ca6820ed031b49a8b3f6cff4ad7a899d`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:10:18.283996 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ba615bc4-1c05-4f0a-a01f-b7e13a63a419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:18.292299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1255/2523] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:10:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:10:18.584976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed22e3-6b4e-4d29-983f-1bb7672da154 service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:18.651563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed22e3-6b4e-4d29-983f-1bb7672da154 service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:18.656016 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04233ff-4327-4908-bb96-736bd66bebab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:18.656670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04233ff-4327-4908-bb96-736bd66bebab service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:18.656943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04233ff-4327-4908-bb96-736bd66bebab service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:18.657210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04233ff-4327-4908-bb96-736bd66bebab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:18.659145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b04233ff-4327-4908-bb96-736bd66bebab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:18.669933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04233ff-4327-4908-bb96-736bd66bebab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:18.671456 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04233ff-4327-4908-bb96-736bd66bebab service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:18.679695 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1269/2524] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:10:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:10:19.010103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba615bc4-1c05-4f0a-a01f-b7e13a63a419 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:19.011768 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ba615bc4-1c05-4f0a-a01f-b7e13a63a419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:19.089239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba615bc4-1c05-4f0a-a01f-b7e13a63a419 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:19.094598 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d477e-c32a-4274-a60d-1cdef4a6555c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:19.095256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d477e-c32a-4274-a60d-1cdef4a6555c service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:19.095535 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d477e-c32a-4274-a60d-1cdef4a6555c service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:19.095813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d477e-c32a-4274-a60d-1cdef4a6555c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:19.097722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1c1d477e-c32a-4274-a60d-1cdef4a6555c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:19.106024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:10:19.111500 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d477e-c32a-4274-a60d-1cdef4a6555c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:19.113454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d477e-c32a-4274-a60d-1cdef4a6555c service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:19.127024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1256/2525] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:10:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:10:20.559084 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2c9c-8a60-49b1-867c-b1409e654692 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:20.559631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2c9c-8a60-49b1-867c-b1409e654692 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:20.559737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2c9c-8a60-49b1-867c-b1409e654692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:20.580247 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-826d2c9c-8a60-49b1-867c-b1409e654692 None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:10:20.581459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-826d2c9c-8a60-49b1-867c-b1409e654692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:20.589997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1270/2526] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:10:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:10:20.603764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d477e-c32a-4274-a60d-1cdef4a6555c service placement] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:20.616594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d477e-c32a-4274-a60d-1cdef4a6555c service placement] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:20.622099 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ec2b8-580c-4d55-9e5e-f256597d723c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:20.622901 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ec2b8-580c-4d55-9e5e-f256597d723c service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:20.623250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ec2b8-580c-4d55-9e5e-f256597d723c service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:20.623531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ec2b8-580c-4d55-9e5e-f256597d723c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:20.633619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ec2b8-580c-4d55-9e5e-f256597d723c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:20.635298 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ec2b8-580c-4d55-9e5e-f256597d723c service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:20.645240 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1257/2527] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:10:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 20 11:10:27.824765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2c9c-8a60-49b1-867c-b1409e654692 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:27.826802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-826d2c9c-8a60-49b1-867c-b1409e654692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:27.905931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2c9c-8a60-49b1-867c-b1409e654692 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:27.911927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b5efe-baab-4c9c-80fc-c121c26f4712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:27.912976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b5efe-baab-4c9c-80fc-c121c26f4712 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:27.913257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b5efe-baab-4c9c-80fc-c121c26f4712 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:27.913706 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b5efe-baab-4c9c-80fc-c121c26f4712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:27.916304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5b8b5efe-baab-4c9c-80fc-c121c26f4712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:27.922445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:10:27.984434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b5efe-baab-4c9c-80fc-c121c26f4712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:27.986383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b5efe-baab-4c9c-80fc-c121c26f4712 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:28.000295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1271/2528] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:10:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:10:30.641141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=9 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:10:34.450556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ec2b8-580c-4d55-9e5e-f256597d723c service swift] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:34.462601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ec2b8-580c-4d55-9e5e-f256597d723c service swift] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:34.473884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082bf00-5ab0-4974-a6db-02d0fd389779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:34.475037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082bf00-5ab0-4974-a6db-02d0fd389779 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:34.475524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082bf00-5ab0-4974-a6db-02d0fd389779 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:34.476007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082bf00-5ab0-4974-a6db-02d0fd389779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:34.479026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f082bf00-5ab0-4974-a6db-02d0fd389779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:34.495480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f082bf00-5ab0-4974-a6db-02d0fd389779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:34.497828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f082bf00-5ab0-4974-a6db-02d0fd389779 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:34.511839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1258/2529] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:10:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:10:35.548319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b5efe-baab-4c9c-80fc-c121c26f4712 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:35.563203 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b5efe-baab-4c9c-80fc-c121c26f4712 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:35.569721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5623a7-8779-43ee-abae-c6d82c676ef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:35.573085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5623a7-8779-43ee-abae-c6d82c676ef8 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:35.573085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5623a7-8779-43ee-abae-c6d82c676ef8 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:35.573085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5623a7-8779-43ee-abae-c6d82c676ef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:35.573564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8c5623a7-8779-43ee-abae-c6d82c676ef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:35.657419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5623a7-8779-43ee-abae-c6d82c676ef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:35.659123 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5623a7-8779-43ee-abae-c6d82c676ef8 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:35.669765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1272/2530] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:10:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:10:35.864488 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082bf00-5ab0-4974-a6db-02d0fd389779 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:35.866957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f082bf00-5ab0-4974-a6db-02d0fd389779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:35.899329 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082bf00-5ab0-4974-a6db-02d0fd389779 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:35.917300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74c923-59f1-4515-9052-8d4ce363bc89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:35.918733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74c923-59f1-4515-9052-8d4ce363bc89 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:35.919946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74c923-59f1-4515-9052-8d4ce363bc89 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:35.921600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74c923-59f1-4515-9052-8d4ce363bc89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:35.927942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6b74c923-59f1-4515-9052-8d4ce363bc89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:35.948773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74c923-59f1-4515-9052-8d4ce363bc89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:35.953849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74c923-59f1-4515-9052-8d4ce363bc89 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:35.983784 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1259/2531] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:10:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:10:41.420295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5623a7-8779-43ee-abae-c6d82c676ef8 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:41.434764 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5623a7-8779-43ee-abae-c6d82c676ef8 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:41.444529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5596277-dbd1-4b18-83c3-92ac217476fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:41.445295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5596277-dbd1-4b18-83c3-92ac217476fc service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:41.445655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5596277-dbd1-4b18-83c3-92ac217476fc service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:41.445975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5596277-dbd1-4b18-83c3-92ac217476fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:41.448241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b5596277-dbd1-4b18-83c3-92ac217476fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:41.467881 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5596277-dbd1-4b18-83c3-92ac217476fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:41.469660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5596277-dbd1-4b18-83c3-92ac217476fc service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:41.483450 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1273/2532] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:10:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:10:45.962501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=12 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:10:51.478655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=59 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:10:54.879918 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74c923-59f1-4515-9052-8d4ce363bc89 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:54.881393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-6b74c923-59f1-4515-9052-8d4ce363bc89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:54.903594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74c923-59f1-4515-9052-8d4ce363bc89 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:54.908962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e636dff-f56e-46bf-a447-a65fdb54a888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:54.909862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e636dff-f56e-46bf-a447-a65fdb54a888 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:54.910207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e636dff-f56e-46bf-a447-a65fdb54a888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:54.910497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e636dff-f56e-46bf-a447-a65fdb54a888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:54.921953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e636dff-f56e-46bf-a447-a65fdb54a888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:54.923483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e636dff-f56e-46bf-a447-a65fdb54a888 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:54.936047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1260/2533] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:10:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:10:58.984547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5596277-dbd1-4b18-83c3-92ac217476fc service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:10:58.997906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5596277-dbd1-4b18-83c3-92ac217476fc service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:10:59.009674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac168c-ddd3-4e7e-92fe-0fd745f96e35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:10:59.010699 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac168c-ddd3-4e7e-92fe-0fd745f96e35 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:10:59.012109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac168c-ddd3-4e7e-92fe-0fd745f96e35 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:10:59.012539 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac168c-ddd3-4e7e-92fe-0fd745f96e35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:10:59.015408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a1ac168c-ddd3-4e7e-92fe-0fd745f96e35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:10:59.034677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac168c-ddd3-4e7e-92fe-0fd745f96e35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:10:59.039296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac168c-ddd3-4e7e-92fe-0fd745f96e35 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:10:59.055660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1274/2534] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:10:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:11:04.929210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=6 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:11:09.041507 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88115d-007b-4910-a982-16a517d1007c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:11:09.042209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88115d-007b-4910-a982-16a517d1007c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:11:09.042209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88115d-007b-4910-a982-16a517d1007c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:11:09.047375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=6 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:11:09.064030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ee88115d-007b-4910-a982-16a517d1007c None None] MFA Rules not processed for user `0c4a4ce86beb47c2b7910264a9238c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:11:09.065238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ee88115d-007b-4910-a982-16a517d1007c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:11:09.072439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1261/2535] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:11:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:11:19.057291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:11:22.702028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac168c-ddd3-4e7e-92fe-0fd745f96e35 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:11:22.706516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a1ac168c-ddd3-4e7e-92fe-0fd745f96e35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:11:22.793610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac168c-ddd3-4e7e-92fe-0fd745f96e35 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:11:22.798459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a9f5-1b4a-4a85-8268-e81d73f3bb3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:11:22.800038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a9f5-1b4a-4a85-8268-e81d73f3bb3b service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:11:22.800301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a9f5-1b4a-4a85-8268-e81d73f3bb3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:11:22.800573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a9f5-1b4a-4a85-8268-e81d73f3bb3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:11:22.802526 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-18e6a9f5-1b4a-4a85-8268-e81d73f3bb3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:11:22.880915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6a9f5-1b4a-4a85-8268-e81d73f3bb3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:11:22.882373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6a9f5-1b4a-4a85-8268-e81d73f3bb3b service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:11:22.893231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1275/2536] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:11:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:11:24.646336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0aaae-a09c-4398-bb67-1c18f048238f None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:11:24.646940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0aaae-a09c-4398-bb67-1c18f048238f None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:11:24.646940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0aaae-a09c-4398-bb67-1c18f048238f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:11:24.666957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-fbc0aaae-a09c-4398-bb67-1c18f048238f None None] MFA Rules not processed for user `ee21ef9eacd24177a852f28d17376191`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:11:24.668364 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fbc0aaae-a09c-4398-bb67-1c18f048238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:11:24.677815 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1262/2537] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:11:25.022904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a9f5-1b4a-4a85-8268-e81d73f3bb3b service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:11:25.033433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a9f5-1b4a-4a85-8268-e81d73f3bb3b service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:11:25.037906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd914c7-bf76-4fd0-8b78-c1faf74f03ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:11:25.038603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd914c7-bf76-4fd0-8b78-c1faf74f03ff service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:11:25.038888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd914c7-bf76-4fd0-8b78-c1faf74f03ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:11:25.039171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd914c7-bf76-4fd0-8b78-c1faf74f03ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:11:25.041360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-acd914c7-bf76-4fd0-8b78-c1faf74f03ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:11:25.052754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd914c7-bf76-4fd0-8b78-c1faf74f03ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:11:25.054229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd914c7-bf76-4fd0-8b78-c1faf74f03ff service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:11:25.063547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1276/2538] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:11:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:11:34.660938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:11:35.059054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=54 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:12:07.763468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c9e0-ead1-443d-b4bd-b95bd0cef019 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:07.764097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c9e0-ead1-443d-b4bd-b95bd0cef019 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:07.764097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4c9e0-ead1-443d-b4bd-b95bd0cef019 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:07.765411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1263/2539] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 11:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:12:07.771830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475ea89-c550-419f-aca9-97c43763a4e7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:07.772332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475ea89-c550-419f-aca9-97c43763a4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:07.772682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475ea89-c550-419f-aca9-97c43763a4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:07.791643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a475ea89-c550-419f-aca9-97c43763a4e7 None None] MFA Rules not processed for user `d73fe529c7d342f895aabcb9a39aba78`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:12:07.792968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a475ea89-c550-419f-aca9-97c43763a4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:07.800110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1277/2540] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:12:07.813089 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4c9e0-ead1-443d-b4bd-b95bd0cef019 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:07.825443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4c9e0-ead1-443d-b4bd-b95bd0cef019 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:07.830279 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f1125-2eed-4fe6-9daf-ebda61b09dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:07.831018 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f1125-2eed-4fe6-9daf-ebda61b09dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:07.831389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f1125-2eed-4fe6-9daf-ebda61b09dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:07.831667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f1125-2eed-4fe6-9daf-ebda61b09dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:07.839974 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f1125-2eed-4fe6-9daf-ebda61b09dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:07.841427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f1125-2eed-4fe6-9daf-ebda61b09dc4 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:07.849587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1264/2541] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:12:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:12:07.935995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3d5ac-c425-4335-ac87-8c1d4da1cba0 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:07.936286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3d5ac-c425-4335-ac87-8c1d4da1cba0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:07.936567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3d5ac-c425-4335-ac87-8c1d4da1cba0 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:07.938408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1278/2542] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 11:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 11:12:07.943994 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf055c-42da-4d1e-983b-55812053bebc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:07.944358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf055c-42da-4d1e-983b-55812053bebc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:07.944700 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf055c-42da-4d1e-983b-55812053bebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:07.989242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4edf055c-42da-4d1e-983b-55812053bebc None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:12:07.990498 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4edf055c-42da-4d1e-983b-55812053bebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:08.002121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1265/2543] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 11:12:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:12:08.022838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3d5ac-c425-4335-ac87-8c1d4da1cba0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:08.034281 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3d5ac-c425-4335-ac87-8c1d4da1cba0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:08.038013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4596787-d620-4172-bf92-9bed5880fbff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:08.038700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4596787-d620-4172-bf92-9bed5880fbff service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:08.038983 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4596787-d620-4172-bf92-9bed5880fbff service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:08.039262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4596787-d620-4172-bf92-9bed5880fbff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:08.050454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4596787-d620-4172-bf92-9bed5880fbff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:08.051905 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4596787-d620-4172-bf92-9bed5880fbff service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:08.063485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1279/2544] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:12:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:12:12.657040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6611963-fba7-481a-8dd7-7f6836d29da8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:12.657568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6611963-fba7-481a-8dd7-7f6836d29da8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:12.657667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6611963-fba7-481a-8dd7-7f6836d29da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:12.676779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b6611963-fba7-481a-8dd7-7f6836d29da8 None None] MFA Rules not processed for user `48e77af2b3a2409fa6d352b0d2674b85`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:12:12.677987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b6611963-fba7-481a-8dd7-7f6836d29da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:12.686067 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1266/2545] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:12:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:12:18.058784 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=9 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:12:22.670811 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=18 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:12:29.118259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4596787-d620-4172-bf92-9bed5880fbff service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:29.130890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4596787-d620-4172-bf92-9bed5880fbff service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:29.137592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff8eb-22af-45e7-b15c-ab1ddf303d12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:29.138303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff8eb-22af-45e7-b15c-ab1ddf303d12 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:29.138590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff8eb-22af-45e7-b15c-ab1ddf303d12 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:29.138864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ff8eb-22af-45e7-b15c-ab1ddf303d12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:29.141687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-792ff8eb-22af-45e7-b15c-ab1ddf303d12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:29.155970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ff8eb-22af-45e7-b15c-ab1ddf303d12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:29.157825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ff8eb-22af-45e7-b15c-ab1ddf303d12 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:29.169446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1280/2546] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:12:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:12:37.458574 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6611963-fba7-481a-8dd7-7f6836d29da8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:37.469556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6611963-fba7-481a-8dd7-7f6836d29da8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:37.474005 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1e7c8-4c1d-4d0a-ba59-e5df91cfdc3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:37.474692 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1e7c8-4c1d-4d0a-ba59-e5df91cfdc3e service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:37.474973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1e7c8-4c1d-4d0a-ba59-e5df91cfdc3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:37.475242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1e7c8-4c1d-4d0a-ba59-e5df91cfdc3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:37.477353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-37d1e7c8-4c1d-4d0a-ba59-e5df91cfdc3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:37.547509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1e7c8-4c1d-4d0a-ba59-e5df91cfdc3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:37.548987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1e7c8-4c1d-4d0a-ba59-e5df91cfdc3e service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:37.559353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1267/2547] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:12:39.164676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=6 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:12:40.652729 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff8eb-22af-45e7-b15c-ab1ddf303d12 service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:40.654294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-792ff8eb-22af-45e7-b15c-ab1ddf303d12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:40.668960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ff8eb-22af-45e7-b15c-ab1ddf303d12 service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:40.674294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f267b29-38ef-4775-9d47-b51e08fe3fbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:40.675033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f267b29-38ef-4775-9d47-b51e08fe3fbb service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:40.675311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f267b29-38ef-4775-9d47-b51e08fe3fbb service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:40.675585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f267b29-38ef-4775-9d47-b51e08fe3fbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:40.677497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6f267b29-38ef-4775-9d47-b51e08fe3fbb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:40.691972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f267b29-38ef-4775-9d47-b51e08fe3fbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:40.693598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f267b29-38ef-4775-9d47-b51e08fe3fbb service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:40.705435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1281/2548] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:12:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:12:40.716100 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1e7c8-4c1d-4d0a-ba59-e5df91cfdc3e service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:40.718673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-37d1e7c8-4c1d-4d0a-ba59-e5df91cfdc3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:40.738877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1e7c8-4c1d-4d0a-ba59-e5df91cfdc3e service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:40.747270 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:40.748239 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:40.748556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:40.748855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] PATH_INFO: `/v3/users/0c4a4ce86beb47c2b7910264a9238c8b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:40.750235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4a4ce86beb47c2b7910264a9238c8b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:40.752770 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:40.805501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4a4ce86beb47c2b7910264a9238c8b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:12:40.806246 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4a4ce86beb47c2b7910264a9238c8b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:12:40.815106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4a4ce86beb47c2b7910264a9238c8b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:12:40.815842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4a4ce86beb47c2b7910264a9238c8b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:12:40.823967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5210d194-6cda-4090-bc48-750ca5724d57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4a4ce86beb47c2b7910264a9238c8b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:12:40.843291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1268/2549] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:12:40 2022] DELETE /identity/v3/users/0c4a4ce86beb47c2b7910264a9238c8b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:12:41.051505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f267b29-38ef-4775-9d47-b51e08fe3fbb service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:41.064112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f267b29-38ef-4775-9d47-b51e08fe3fbb service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:41.129562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:41.130466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.130754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.131093 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] PATH_INFO: `/v3/projects/4cab4c10ddc745418a470e5d50b77bea` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.132262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cab4c10ddc745418a470e5d50b77bea)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:41.134227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:41.183799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cab4c10ddc745418a470e5d50b77bea', 'request_id': 'req-7fa72218-6288-4fa3-b7d6-a0a186a9dead'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:12:41.192834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cab4c10ddc745418a470e5d50b77bea', 'request_id': 'req-7fa72218-6288-4fa3-b7d6-a0a186a9dead'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:12:41.199278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cab4c10ddc745418a470e5d50b77bea', 'request_id': 'req-7fa72218-6288-4fa3-b7d6-a0a186a9dead'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:12:41.213502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] The token cache is being invalidate because project 4cab4c10ddc745418a470e5d50b77bea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:12:41.213794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:12:41.214513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7fa72218-6288-4fa3-b7d6-a0a186a9dead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:12:41.216079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1282/2550] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:12:41 2022] DELETE /identity/v3/projects/4cab4c10ddc745418a470e5d50b77bea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:12:41.221768 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517aa15c-eeba-48fc-9db8-583a98c7eea6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.222006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517aa15c-eeba-48fc-9db8-583a98c7eea6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.222303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517aa15c-eeba-48fc-9db8-583a98c7eea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.240702 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-517aa15c-eeba-48fc-9db8-583a98c7eea6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:12:41.241996 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-517aa15c-eeba-48fc-9db8-583a98c7eea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:41.304892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1269/2551] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:12:41.313312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48184f35-7fe2-4855-87bf-aed0ad101ea6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.313631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48184f35-7fe2-4855-87bf-aed0ad101ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.313934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48184f35-7fe2-4855-87bf-aed0ad101ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.333725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-48184f35-7fe2-4855-87bf-aed0ad101ea6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:12:41.335007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-48184f35-7fe2-4855-87bf-aed0ad101ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:41.344325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1283/2552] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:12:41.348940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517aa15c-eeba-48fc-9db8-583a98c7eea6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:41.359554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517aa15c-eeba-48fc-9db8-583a98c7eea6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:41.365027 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48658aa9-636c-4e61-a602-b0714c7e8c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:41.365699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48658aa9-636c-4e61-a602-b0714c7e8c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.365981 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48658aa9-636c-4e61-a602-b0714c7e8c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.366286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48658aa9-636c-4e61-a602-b0714c7e8c5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.367066 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48658aa9-636c-4e61-a602-b0714c7e8c5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:12:41.367448 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48658aa9-636c-4e61-a602-b0714c7e8c5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:41.369276 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48658aa9-636c-4e61-a602-b0714c7e8c5d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:41.370922 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:12:41.386951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1270/2553] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:12:41.388120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48184f35-7fe2-4855-87bf-aed0ad101ea6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:41.397852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48184f35-7fe2-4855-87bf-aed0ad101ea6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:41.404012 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758453-f71d-4850-b0bb-7946c2cee180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:41.404683 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758453-f71d-4850-b0bb-7946c2cee180 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.404965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758453-f71d-4850-b0bb-7946c2cee180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.405237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1758453-f71d-4850-b0bb-7946c2cee180 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.405985 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1758453-f71d-4850-b0bb-7946c2cee180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:41.407949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1758453-f71d-4850-b0bb-7946c2cee180 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:41.446342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1284/2554] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:12:41 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:12:41.450814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48658aa9-636c-4e61-a602-b0714c7e8c5d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:41.461665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48658aa9-636c-4e61-a602-b0714c7e8c5d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:41.522367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1c45e-6ace-4298-881f-4f337f7a18f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:41.523054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1c45e-6ace-4298-881f-4f337f7a18f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.523451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1c45e-6ace-4298-881f-4f337f7a18f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.523743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1c45e-6ace-4298-881f-4f337f7a18f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.524605 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1c45e-6ace-4298-881f-4f337f7a18f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:41.526595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1c45e-6ace-4298-881f-4f337f7a18f4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:41.554959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1271/2555] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:12:41 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:12:41.558386 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758453-f71d-4850-b0bb-7946c2cee180 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:41.570151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1758453-f71d-4850-b0bb-7946c2cee180 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:41.575059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ec0a8-d21d-4322-bd32-e56b4e2ac68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:41.575737 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ec0a8-d21d-4322-bd32-e56b4e2ac68e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.576019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ec0a8-d21d-4322-bd32-e56b4e2ac68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.576294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ec0a8-d21d-4322-bd32-e56b4e2ac68e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.577023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ec0a8-d21d-4322-bd32-e56b4e2ac68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:41.578802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ec0a8-d21d-4322-bd32-e56b4e2ac68e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:41.593569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1285/2556] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:12:41.595084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1c45e-6ace-4298-881f-4f337f7a18f4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:41.605668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1c45e-6ace-4298-881f-4f337f7a18f4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:41.611814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c2eb5-7535-4d09-959d-b518dac8b725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:41.612506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c2eb5-7535-4d09-959d-b518dac8b725 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.612999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c2eb5-7535-4d09-959d-b518dac8b725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.613282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c2eb5-7535-4d09-959d-b518dac8b725 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.614025 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c2eb5-7535-4d09-959d-b518dac8b725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:41.615699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c2eb5-7535-4d09-959d-b518dac8b725 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:41.628210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1272/2557] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:12:41.632387 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ec0a8-d21d-4322-bd32-e56b4e2ac68e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:41.643514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ec0a8-d21d-4322-bd32-e56b4e2ac68e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:41.648279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbdd2-c268-47bc-95d3-22bcd8c739f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:41.649538 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbdd2-c268-47bc-95d3-22bcd8c739f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.649819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbdd2-c268-47bc-95d3-22bcd8c739f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.650112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbdd2-c268-47bc-95d3-22bcd8c739f1 admin admin] PATH_INFO: `/v3/projects/5778b58ba0bb4bebb334d08971c67323/users/b7275026b00b4e93bed7682b3ff2525c/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.666106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fbdd2-c268-47bc-95d3-22bcd8c739f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5778b58ba0bb4bebb334d08971c67323, user_id=b7275026b00b4e93bed7682b3ff2525c, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:41.669101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fbdd2-c268-47bc-95d3-22bcd8c739f1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:41.682716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1286/2558] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:12:41 2022] PUT /identity/v3/projects/5778b58ba0bb4bebb334d08971c67323/users/b7275026b00b4e93bed7682b3ff2525c/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:12:41.688443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653fb47-8df9-4c6e-a960-5152a88030d1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.688731 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653fb47-8df9-4c6e-a960-5152a88030d1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.689005 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653fb47-8df9-4c6e-a960-5152a88030d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.707000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7653fb47-8df9-4c6e-a960-5152a88030d1 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:12:41.708212 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7653fb47-8df9-4c6e-a960-5152a88030d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:41.772086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1273/2559] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:12:41.782232 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbdd2-c268-47bc-95d3-22bcd8c739f1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:41.784275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-004fbdd2-c268-47bc-95d3-22bcd8c739f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:41.791754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:12:41.845674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbdd2-c268-47bc-95d3-22bcd8c739f1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:41.850194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca55015-7bd8-4120-8cfd-2fb6767aa894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:41.850863 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca55015-7bd8-4120-8cfd-2fb6767aa894 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:41.851141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca55015-7bd8-4120-8cfd-2fb6767aa894 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:41.851468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca55015-7bd8-4120-8cfd-2fb6767aa894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:41.862462 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca55015-7bd8-4120-8cfd-2fb6767aa894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:41.863909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca55015-7bd8-4120-8cfd-2fb6767aa894 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:41.874812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1287/2560] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:12:45.914047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb9255-e6a3-4955-824f-004d987e4709 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:45.914683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb9255-e6a3-4955-824f-004d987e4709 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:45.914683 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb9255-e6a3-4955-824f-004d987e4709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:45.944376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dfdb9255-e6a3-4955-824f-004d987e4709 None None] MFA Rules not processed for user `b7275026b00b4e93bed7682b3ff2525c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:12:45.946007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dfdb9255-e6a3-4955-824f-004d987e4709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:46.027715 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1274/2561] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:12:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:12:46.034747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca55015-7bd8-4120-8cfd-2fb6767aa894 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:12:46.036171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8ca55015-7bd8-4120-8cfd-2fb6767aa894 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:46.113370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca55015-7bd8-4120-8cfd-2fb6767aa894 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:12:46.119856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27d3f3-c9d4-4441-87a4-602f13011ca4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:12:46.119856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27d3f3-c9d4-4441-87a4-602f13011ca4 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:46.119856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27d3f3-c9d4-4441-87a4-602f13011ca4 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:46.120184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27d3f3-c9d4-4441-87a4-602f13011ca4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:46.129541 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27d3f3-c9d4-4441-87a4-602f13011ca4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:12:46.131597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27d3f3-c9d4-4441-87a4-602f13011ca4 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:12:46.162593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1288/2562] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:12:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:12:47.738175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2cd52-ca9c-4253-840c-48eac1077362 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:12:47.738625 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2cd52-ca9c-4253-840c-48eac1077362 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:12:47.738835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2cd52-ca9c-4253-840c-48eac1077362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:12:47.745607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:12:47.761830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-26d2cd52-ca9c-4253-840c-48eac1077362 None None] MFA Rules not processed for user `b7275026b00b4e93bed7682b3ff2525c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:12:47.763088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-26d2cd52-ca9c-4253-840c-48eac1077362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:12:47.770226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1275/2563] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:12:56.145527 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=58 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:12:57.758807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:13:45.249602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5a09-8b4e-413d-88d2-f0619aa023ca None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:13:45.250212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5a09-8b4e-413d-88d2-f0619aa023ca None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:13:45.250352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5a09-8b4e-413d-88d2-f0619aa023ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:13:45.268795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-318f5a09-8b4e-413d-88d2-f0619aa023ca None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:13:45.269984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-318f5a09-8b4e-413d-88d2-f0619aa023ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:13:45.327052 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1289/2564] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jun 20 11:13:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:13:45.373971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2cd52-ca9c-4253-840c-48eac1077362 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:13:45.375969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-26d2cd52-ca9c-4253-840c-48eac1077362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:13:45.448817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2cd52-ca9c-4253-840c-48eac1077362 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:13:45.454336 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179edf17-5d5c-4db2-aa49-3695ceabd697 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:13:45.455723 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179edf17-5d5c-4db2-aa49-3695ceabd697 service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:13:45.456084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179edf17-5d5c-4db2-aa49-3695ceabd697 service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:13:45.456383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179edf17-5d5c-4db2-aa49-3695ceabd697 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:13:45.466306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179edf17-5d5c-4db2-aa49-3695ceabd697 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:13:45.467968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179edf17-5d5c-4db2-aa49-3695ceabd697 service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:13:45.483750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1276/2565] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:13:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:13:50.864883 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53fcf1-6649-4286-8f89-624ca452563d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:13:50.865483 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53fcf1-6649-4286-8f89-624ca452563d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:13:50.865959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53fcf1-6649-4286-8f89-624ca452563d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:13:50.887442 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-fa53fcf1-6649-4286-8f89-624ca452563d None None] MFA Rules not processed for user `ee21ef9eacd24177a852f28d17376191`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:13:50.889253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fa53fcf1-6649-4286-8f89-624ca452563d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:13:50.960766 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1290/2566] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:13:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:13:52.172832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179edf17-5d5c-4db2-aa49-3695ceabd697 service swift] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:13:52.174620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-179edf17-5d5c-4db2-aa49-3695ceabd697 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:13:52.191349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179edf17-5d5c-4db2-aa49-3695ceabd697 service swift] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:13:52.199502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:13:52.200441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:13:52.200689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:13:52.200966 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] PATH_INFO: `/v3/users/ca6820ed031b49a8b3f6cff4ad7a899d` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:13:52.202218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6820ed031b49a8b3f6cff4ad7a899d)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:13:52.204241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:13:52.257763 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6820ed031b49a8b3f6cff4ad7a899d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.258527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6820ed031b49a8b3f6cff4ad7a899d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.267626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6820ed031b49a8b3f6cff4ad7a899d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.268161 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6820ed031b49a8b3f6cff4ad7a899d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.276164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6820ed031b49a8b3f6cff4ad7a899d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.294064 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1277/2567] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:13:52 2022] DELETE /identity/v3/users/ca6820ed031b49a8b3f6cff4ad7a899d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:13:52.481839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53fcf1-6649-4286-8f89-624ca452563d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:13:52.498385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53fcf1-6649-4286-8f89-624ca452563d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:13:52.567518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:13:52.568410 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:13:52.568687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:13:52.568961 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] PATH_INFO: `/v3/projects/fdaa800381f948619a950d65f637c16a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:13:52.570183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdaa800381f948619a950d65f637c16a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:13:52.572143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:13:52.620300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdaa800381f948619a950d65f637c16a', 'request_id': 'req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.622775 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:13:52.628506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdaa800381f948619a950d65f637c16a', 'request_id': 'req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.635245 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdaa800381f948619a950d65f637c16a', 'request_id': 'req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.646443 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] The token cache is being invalidate because project fdaa800381f948619a950d65f637c16a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:13:52.646729 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.647454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.649120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1291/2568] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:13:52 2022] DELETE /identity/v3/projects/fdaa800381f948619a950d65f637c16a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:13:52.653354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:13:52.654908 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:13:52.668511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302331fc-9bba-4fd1-a4eb-9be864d24150 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:13:52.725392 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:13:52.726333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:13:52.726629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:13:52.726912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] PATH_INFO: `/v3/users/3ef07f57ca4041baae444ec89958c62f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:13:52.728202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ef07f57ca4041baae444ec89958c62f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:13:52.730197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:13:52.732611 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:13:52.792089 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ef07f57ca4041baae444ec89958c62f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.792910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ef07f57ca4041baae444ec89958c62f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.800694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ef07f57ca4041baae444ec89958c62f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.801473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ef07f57ca4041baae444ec89958c62f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.810643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ef07f57ca4041baae444ec89958c62f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:52.830859 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1278/2569] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:13:52 2022] DELETE /identity/v3/users/3ef07f57ca4041baae444ec89958c62f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:13:53.095112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:13:53.097055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:13:53.127906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff792d-b559-4aac-a7f8-408e8ed9dc2f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:13:53.190779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:13:53.191675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:13:53.192135 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:13:53.193054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] PATH_INFO: `/v3/projects/1d2977bee89c4b65938cfb896c02fbee` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:13:53.194850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2977bee89c4b65938cfb896c02fbee)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:13:53.196763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:13:53.263606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2977bee89c4b65938cfb896c02fbee', 'request_id': 'req-b32faed7-086e-469e-8651-b0c1979f83a0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:53.275406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2977bee89c4b65938cfb896c02fbee', 'request_id': 'req-b32faed7-086e-469e-8651-b0c1979f83a0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:53.282272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2977bee89c4b65938cfb896c02fbee', 'request_id': 'req-b32faed7-086e-469e-8651-b0c1979f83a0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:53.295028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] The token cache is being invalidate because project 1d2977bee89c4b65938cfb896c02fbee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:13:53.295448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:53.296142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:13:53.298064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1292/2570] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:13:53 2022] DELETE /identity/v3/projects/1d2977bee89c4b65938cfb896c02fbee => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:02.826575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:03.291700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:07.882580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:07.884857 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:07.956795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bdee0-a0d6-4e21-b0de-ab5208dce988 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:08.011203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593b89f-b1e0-4000-a50d-c0646c29a43b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:08.012000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593b89f-b1e0-4000-a50d-c0646c29a43b service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:08.012272 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593b89f-b1e0-4000-a50d-c0646c29a43b service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:08.012579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593b89f-b1e0-4000-a50d-c0646c29a43b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:08.014747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f593b89f-b1e0-4000-a50d-c0646c29a43b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:08.079408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593b89f-b1e0-4000-a50d-c0646c29a43b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:08.081999 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593b89f-b1e0-4000-a50d-c0646c29a43b service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:08.094725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1279/2571] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:14:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:14:10.186924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:10.188434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:10.201709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32faed7-086e-469e-8651-b0c1979f83a0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:10.205640 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cf492-c278-411d-9ad9-1f8b75662d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:10.206563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cf492-c278-411d-9ad9-1f8b75662d92 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:10.206779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cf492-c278-411d-9ad9-1f8b75662d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:10.207055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cf492-c278-411d-9ad9-1f8b75662d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:10.209031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6a3cf492-c278-411d-9ad9-1f8b75662d92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:10.289214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cf492-c278-411d-9ad9-1f8b75662d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:10.290768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cf492-c278-411d-9ad9-1f8b75662d92 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:10.301153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1293/2572] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:14:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:14:14.236971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593b89f-b1e0-4000-a50d-c0646c29a43b service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:14.249367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593b89f-b1e0-4000-a50d-c0646c29a43b service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:14.253241 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b10a5f-4d67-40cd-84d3-b0f023356e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:14.253985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b10a5f-4d67-40cd-84d3-b0f023356e2a service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:14.254234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b10a5f-4d67-40cd-84d3-b0f023356e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:14.254516 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b10a5f-4d67-40cd-84d3-b0f023356e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:14.256545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e2b10a5f-4d67-40cd-84d3-b0f023356e2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:14.331446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b10a5f-4d67-40cd-84d3-b0f023356e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:14.332920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b10a5f-4d67-40cd-84d3-b0f023356e2a service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:14.343792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1280/2573] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:14:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:14:20.297023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=30 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:20.864340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cf492-c278-411d-9ad9-1f8b75662d92 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:20.865915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6a3cf492-c278-411d-9ad9-1f8b75662d92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:20.881229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cf492-c278-411d-9ad9-1f8b75662d92 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:20.886196 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:20.887139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:20.887422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:20.887704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] PATH_INFO: `/v3/users/fdff9a743af948f88d60a6fd80795c06` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:20.889097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdff9a743af948f88d60a6fd80795c06)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:20.891194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:20.942948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdff9a743af948f88d60a6fd80795c06'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:20.951567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdff9a743af948f88d60a6fd80795c06'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:20.952169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdff9a743af948f88d60a6fd80795c06'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:20.952789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdff9a743af948f88d60a6fd80795c06'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:20.965652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdff9a743af948f88d60a6fd80795c06'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:20.979051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1294/2574] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:14:20 2022] DELETE /identity/v3/users/fdff9a743af948f88d60a6fd80795c06 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:21.181495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b10a5f-4d67-40cd-84d3-b0f023356e2a service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:21.189816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:21.195327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b10a5f-4d67-40cd-84d3-b0f023356e2a service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:21.254114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:21.255008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:21.255289 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:21.255564 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] PATH_INFO: `/v3/projects/51c3f69c523b49e68d7d4d09f3236c4e` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:21.256752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c3f69c523b49e68d7d4d09f3236c4e)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:21.258785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:21.306984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c3f69c523b49e68d7d4d09f3236c4e', 'request_id': 'req-580b105c-a133-4cea-a53d-0b1337c4b5f7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.315199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c3f69c523b49e68d7d4d09f3236c4e', 'request_id': 'req-580b105c-a133-4cea-a53d-0b1337c4b5f7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.320667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c3f69c523b49e68d7d4d09f3236c4e', 'request_id': 'req-580b105c-a133-4cea-a53d-0b1337c4b5f7'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.331968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] The token cache is being invalidate because project 51c3f69c523b49e68d7d4d09f3236c4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:14:21.332228 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.332945 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.334946 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1281/2575] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:14:21 2022] DELETE /identity/v3/projects/51c3f69c523b49e68d7d4d09f3236c4e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:21.338763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:21.340655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:21.353757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70dab6-7880-4ad8-b775-d7eeba58da76 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:21.414445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:21.415336 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:21.415616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:21.415924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] PATH_INFO: `/v3/users/4ea5ff9a8c45438f980d1b1755e5b815` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:21.417285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ea5ff9a8c45438f980d1b1755e5b815)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:21.419309 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:21.473513 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ea5ff9a8c45438f980d1b1755e5b815'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.483335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ea5ff9a8c45438f980d1b1755e5b815'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.483971 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ea5ff9a8c45438f980d1b1755e5b815'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.484612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ea5ff9a8c45438f980d1b1755e5b815'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.497566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ea5ff9a8c45438f980d1b1755e5b815'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.499674 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:21.513710 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1295/2576] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:14:21 2022] DELETE /identity/v3/users/4ea5ff9a8c45438f980d1b1755e5b815 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:14:21.703255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:21.704889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:21.723896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b105c-a133-4cea-a53d-0b1337c4b5f7 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:21.786443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:21.787341 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:21.787614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:21.787883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] PATH_INFO: `/v3/projects/e7a519fd2119461cb1985e6d3752b212` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:21.789124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a519fd2119461cb1985e6d3752b212)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:21.791098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:21.832382 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:21.852395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a519fd2119461cb1985e6d3752b212', 'request_id': 'req-a85a5081-db5e-4663-9d4d-a37d9ddd141f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.861900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a519fd2119461cb1985e6d3752b212', 'request_id': 'req-a85a5081-db5e-4663-9d4d-a37d9ddd141f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.868597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a519fd2119461cb1985e6d3752b212', 'request_id': 'req-a85a5081-db5e-4663-9d4d-a37d9ddd141f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.882168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] The token cache is being invalidate because project e7a519fd2119461cb1985e6d3752b212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:14:21.882484 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.883134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:21.884698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1282/2577] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:14:21 2022] DELETE /identity/v3/projects/e7a519fd2119461cb1985e6d3752b212 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:22.118166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:22.119839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:22.190807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cc6b8-0495-4a3a-83af-78c3e9c4cff3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:22.246532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45574fa9-dde6-4be0-8d65-2a4175814ea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:22.247198 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574fa9-dde6-4be0-8d65-2a4175814ea7 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:22.247556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574fa9-dde6-4be0-8d65-2a4175814ea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:22.247821 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45574fa9-dde6-4be0-8d65-2a4175814ea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:22.249972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-45574fa9-dde6-4be0-8d65-2a4175814ea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:22.328297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45574fa9-dde6-4be0-8d65-2a4175814ea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:22.329809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45574fa9-dde6-4be0-8d65-2a4175814ea7 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:22.341719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1296/2578] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:14:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:14:27.745455 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:27.747143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:27.809953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a5081-db5e-4663-9d4d-a37d9ddd141f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:27.814042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f9f91-5161-4800-9382-4e188d77e6e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:27.814830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f9f91-5161-4800-9382-4e188d77e6e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:27.815157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f9f91-5161-4800-9382-4e188d77e6e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:27.815489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f9f91-5161-4800-9382-4e188d77e6e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:27.817510 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-234f9f91-5161-4800-9382-4e188d77e6e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:27.833451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f9f91-5161-4800-9382-4e188d77e6e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:27.834950 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f9f91-5161-4800-9382-4e188d77e6e0 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:27.847787 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1283/2579] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:14:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:14:32.336586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=78 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:35.626841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45574fa9-dde6-4be0-8d65-2a4175814ea7 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:35.637923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45574fa9-dde6-4be0-8d65-2a4175814ea7 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:35.643416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857e669-b75b-4921-8631-6f1c2b093a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:35.644491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857e669-b75b-4921-8631-6f1c2b093a94 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:35.644949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857e669-b75b-4921-8631-6f1c2b093a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:35.645329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857e669-b75b-4921-8631-6f1c2b093a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:35.648592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2857e669-b75b-4921-8631-6f1c2b093a94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:35.721585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857e669-b75b-4921-8631-6f1c2b093a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:35.723075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857e669-b75b-4921-8631-6f1c2b093a94 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:35.733557 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1297/2580] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:14:37.841465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=37,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:39.123393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f9f91-5161-4800-9382-4e188d77e6e0 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:39.123393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-234f9f91-5161-4800-9382-4e188d77e6e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:39.138186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f9f91-5161-4800-9382-4e188d77e6e0 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:39.146560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:39.147479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:39.148454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:39.148747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] PATH_INFO: `/v3/users/48e77af2b3a2409fa6d352b0d2674b85` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:39.168549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e77af2b3a2409fa6d352b0d2674b85)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:39.170733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:39.236186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e77af2b3a2409fa6d352b0d2674b85'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:39.237022 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e77af2b3a2409fa6d352b0d2674b85'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:39.245625 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e77af2b3a2409fa6d352b0d2674b85'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:39.246408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e77af2b3a2409fa6d352b0d2674b85'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:39.257338 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-ef132b21-4002-4301-a194-95e3126bad5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e77af2b3a2409fa6d352b0d2674b85'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:39.277887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1284/2581] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:14:39 2022] DELETE /identity/v3/users/48e77af2b3a2409fa6d352b0d2674b85 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:39.555619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857e669-b75b-4921-8631-6f1c2b093a94 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:39.577293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857e669-b75b-4921-8631-6f1c2b093a94 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:39.639175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:39.640081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:39.640356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:39.640629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] PATH_INFO: `/v3/projects/7eb9dd7f67cb48cebba2dabcd32a21d7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:39.652651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eb9dd7f67cb48cebba2dabcd32a21d7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:39.654625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:39.704992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eb9dd7f67cb48cebba2dabcd32a21d7', 'request_id': 'req-43b7ec0a-3902-4088-a383-d247820afc92'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:39.714188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eb9dd7f67cb48cebba2dabcd32a21d7', 'request_id': 'req-43b7ec0a-3902-4088-a383-d247820afc92'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:39.721347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eb9dd7f67cb48cebba2dabcd32a21d7', 'request_id': 'req-43b7ec0a-3902-4088-a383-d247820afc92'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:39.735083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] The token cache is being invalidate because project 7eb9dd7f67cb48cebba2dabcd32a21d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:14:39.735379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:39.736126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-43b7ec0a-3902-4088-a383-d247820afc92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:39.737820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1298/2582] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:14:39 2022] DELETE /identity/v3/projects/7eb9dd7f67cb48cebba2dabcd32a21d7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:39.743054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1e6d5-6cc4-4e35-9d09-a66fda9c15c9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:39.743351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1e6d5-6cc4-4e35-9d09-a66fda9c15c9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:39.743631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1e6d5-6cc4-4e35-9d09-a66fda9c15c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:39.762372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-86b1e6d5-6cc4-4e35-9d09-a66fda9c15c9 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:14:39.763656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-86b1e6d5-6cc4-4e35-9d09-a66fda9c15c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:39.826747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1285/2583] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:14:39.834719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c846f-e7b8-4654-a4d0-00d983c5c8cf None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:39.835327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c846f-e7b8-4654-a4d0-00d983c5c8cf None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:39.835637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c846f-e7b8-4654-a4d0-00d983c5c8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:39.854130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-956c846f-e7b8-4654-a4d0-00d983c5c8cf None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:14:39.855562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-956c846f-e7b8-4654-a4d0-00d983c5c8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:39.863962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1299/2584] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:14:39.868399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1e6d5-6cc4-4e35-9d09-a66fda9c15c9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:39.879355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1e6d5-6cc4-4e35-9d09-a66fda9c15c9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:39.884753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951d246-e3da-49c3-bd6c-dea9ff756854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:39.885395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951d246-e3da-49c3-bd6c-dea9ff756854 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:39.885931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951d246-e3da-49c3-bd6c-dea9ff756854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:39.886599 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951d246-e3da-49c3-bd6c-dea9ff756854 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:39.887339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951d246-e3da-49c3-bd6c-dea9ff756854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:14:39.887650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951d246-e3da-49c3-bd6c-dea9ff756854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:39.889309 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951d246-e3da-49c3-bd6c-dea9ff756854 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:39.902012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1286/2585] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:14:39.906645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c846f-e7b8-4654-a4d0-00d983c5c8cf None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:39.917139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c846f-e7b8-4654-a4d0-00d983c5c8cf None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:39.923061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da971b3-8e5f-4813-9b8a-d7f0ae2786b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:39.923736 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da971b3-8e5f-4813-9b8a-d7f0ae2786b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:39.924042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da971b3-8e5f-4813-9b8a-d7f0ae2786b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:39.924319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da971b3-8e5f-4813-9b8a-d7f0ae2786b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:39.925013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da971b3-8e5f-4813-9b8a-d7f0ae2786b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:39.926980 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da971b3-8e5f-4813-9b8a-d7f0ae2786b7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:39.931675 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:39.958838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1300/2586] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:14:39 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:14:39.962629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951d246-e3da-49c3-bd6c-dea9ff756854 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:39.974441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951d246-e3da-49c3-bd6c-dea9ff756854 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:40.021402 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:40.036810 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7240728-d9c3-4f90-b77b-39fb8b09fb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:40.037501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7240728-d9c3-4f90-b77b-39fb8b09fb04 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:40.037788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7240728-d9c3-4f90-b77b-39fb8b09fb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:40.038070 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7240728-d9c3-4f90-b77b-39fb8b09fb04 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:40.038798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7240728-d9c3-4f90-b77b-39fb8b09fb04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:40.040753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7240728-d9c3-4f90-b77b-39fb8b09fb04 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:40.063186 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1287/2587] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:14:39 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:14:40.067598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da971b3-8e5f-4813-9b8a-d7f0ae2786b7 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:40.078604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da971b3-8e5f-4813-9b8a-d7f0ae2786b7 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:40.083895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965a8b1-499b-4f34-91c6-a3ae50bcf4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:40.084568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965a8b1-499b-4f34-91c6-a3ae50bcf4ba admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:40.084857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965a8b1-499b-4f34-91c6-a3ae50bcf4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:40.085134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965a8b1-499b-4f34-91c6-a3ae50bcf4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:40.085868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965a8b1-499b-4f34-91c6-a3ae50bcf4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:40.087530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965a8b1-499b-4f34-91c6-a3ae50bcf4ba admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:40.102482 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1301/2588] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:14:40.109180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7240728-d9c3-4f90-b77b-39fb8b09fb04 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:40.120927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7240728-d9c3-4f90-b77b-39fb8b09fb04 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:40.131585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca9708-befa-4d96-abe7-674b7b66ff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:40.132398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca9708-befa-4d96-abe7-674b7b66ff68 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:40.132728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca9708-befa-4d96-abe7-674b7b66ff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:40.133055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca9708-befa-4d96-abe7-674b7b66ff68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:40.133909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca9708-befa-4d96-abe7-674b7b66ff68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:40.135848 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca9708-befa-4d96-abe7-674b7b66ff68 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:40.147465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1288/2589] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:14:40.152140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965a8b1-499b-4f34-91c6-a3ae50bcf4ba admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:40.163169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965a8b1-499b-4f34-91c6-a3ae50bcf4ba admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:40.168490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311352ff-bc94-46f1-8200-d48b8b07b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:40.169392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311352ff-bc94-46f1-8200-d48b8b07b575 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:40.169670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311352ff-bc94-46f1-8200-d48b8b07b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:40.169929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311352ff-bc94-46f1-8200-d48b8b07b575 admin admin] PATH_INFO: `/v3/projects/62c73564fd214b0381afbc25ac63e4d9/users/1a75665a44c049ce8d218c9c81024567/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:40.187668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311352ff-bc94-46f1-8200-d48b8b07b575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c73564fd214b0381afbc25ac63e4d9, user_id=1a75665a44c049ce8d218c9c81024567, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:40.190351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311352ff-bc94-46f1-8200-d48b8b07b575 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:40.204445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1302/2590] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:14:40 2022] PUT /identity/v3/projects/62c73564fd214b0381afbc25ac63e4d9/users/1a75665a44c049ce8d218c9c81024567/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:40.215801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f461153-44fe-43df-b817-b3599c2c2f67 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:40.216076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f461153-44fe-43df-b817-b3599c2c2f67 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:40.216351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f461153-44fe-43df-b817-b3599c2c2f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:40.245235 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4f461153-44fe-43df-b817-b3599c2c2f67 None None] MFA Rules not processed for user `1a75665a44c049ce8d218c9c81024567`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:14:40.246531 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4f461153-44fe-43df-b817-b3599c2c2f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:40.323415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1289/2591] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:14:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:14:40.335178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311352ff-bc94-46f1-8200-d48b8b07b575 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:40.336982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-311352ff-bc94-46f1-8200-d48b8b07b575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:40.421207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311352ff-bc94-46f1-8200-d48b8b07b575 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:40.427269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f8d32-7171-4789-8002-bb6658120615 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:40.427931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f8d32-7171-4789-8002-bb6658120615 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:40.428214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f8d32-7171-4789-8002-bb6658120615 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:40.428488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f8d32-7171-4789-8002-bb6658120615 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:40.437416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f8d32-7171-4789-8002-bb6658120615 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:40.439052 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f8d32-7171-4789-8002-bb6658120615 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:40.448823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1303/2592] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:14:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:14:50.319689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:50.444284 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:50.946905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f461153-44fe-43df-b817-b3599c2c2f67 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:51.034132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f461153-44fe-43df-b817-b3599c2c2f67 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:51.039780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:51.041138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:51.041422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:51.041757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] PATH_INFO: `/v3/users/1a75665a44c049ce8d218c9c81024567` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:51.043068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a75665a44c049ce8d218c9c81024567)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:51.045093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:51.123830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a75665a44c049ce8d218c9c81024567'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:51.124482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a75665a44c049ce8d218c9c81024567'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:51.134621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a75665a44c049ce8d218c9c81024567'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:51.135236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a75665a44c049ce8d218c9c81024567'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:51.143996 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a75665a44c049ce8d218c9c81024567'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:51.167196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1290/2593] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:14:50 2022] DELETE /identity/v3/users/1a75665a44c049ce8d218c9c81024567 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:51.173015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ff250-20a7-439f-b2f4-45ebddef0116 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:51.173719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ff250-20a7-439f-b2f4-45ebddef0116 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:51.173719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ff250-20a7-439f-b2f4-45ebddef0116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:51.193107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-939ff250-20a7-439f-b2f4-45ebddef0116 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:14:51.194369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-939ff250-20a7-439f-b2f4-45ebddef0116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:51.262018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1304/2594] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:14:51.273498 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:51.275251 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:51.350058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b350256-fa13-4453-84f2-64a9b520122f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:51.357789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7ef2f-177e-42be-95b9-5c31d00e429c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:51.358489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7ef2f-177e-42be-95b9-5c31d00e429c service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:51.358758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7ef2f-177e-42be-95b9-5c31d00e429c service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:51.359029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7ef2f-177e-42be-95b9-5c31d00e429c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:51.370183 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7ef2f-177e-42be-95b9-5c31d00e429c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:51.371671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7ef2f-177e-42be-95b9-5c31d00e429c service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:51.382939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1291/2595] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:14:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:14:51.689530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ff250-20a7-439f-b2f4-45ebddef0116 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:51.691126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-939ff250-20a7-439f-b2f4-45ebddef0116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:51.703979 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ff250-20a7-439f-b2f4-45ebddef0116 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:51.709796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:51.710796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:51.711077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:51.711352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] PATH_INFO: `/v3/projects/62c73564fd214b0381afbc25ac63e4d9` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:51.712668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62c73564fd214b0381afbc25ac63e4d9)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:51.714634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:51.769033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62c73564fd214b0381afbc25ac63e4d9', 'request_id': 'req-3c1cb9b2-dad2-479f-8556-c416bd6fb479'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:51.776673 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62c73564fd214b0381afbc25ac63e4d9', 'request_id': 'req-3c1cb9b2-dad2-479f-8556-c416bd6fb479'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:51.782578 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62c73564fd214b0381afbc25ac63e4d9', 'request_id': 'req-3c1cb9b2-dad2-479f-8556-c416bd6fb479'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:51.794760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] The token cache is being invalidate because project 62c73564fd214b0381afbc25ac63e4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:14:51.795052 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:51.795816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3c1cb9b2-dad2-479f-8556-c416bd6fb479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:14:51.797591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1305/2596] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:14:51 2022] DELETE /identity/v3/projects/62c73564fd214b0381afbc25ac63e4d9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:51.804842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37be04-3d6c-4bb5-ab51-ffd1466268e9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:51.805320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37be04-3d6c-4bb5-ab51-ffd1466268e9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:51.805765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37be04-3d6c-4bb5-ab51-ffd1466268e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:51.828051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-cf37be04-3d6c-4bb5-ab51-ffd1466268e9 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:14:51.829320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-cf37be04-3d6c-4bb5-ab51-ffd1466268e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:51.855871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:51.904851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1292/2597] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:14:51.913647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343e456-d154-4613-99c9-f80e3eed8d16 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:51.913944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343e456-d154-4613-99c9-f80e3eed8d16 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:51.914257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343e456-d154-4613-99c9-f80e3eed8d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:51.933682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4343e456-d154-4613-99c9-f80e3eed8d16 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:14:51.935212 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4343e456-d154-4613-99c9-f80e3eed8d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:51.943929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1306/2598] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:14:51.948506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37be04-3d6c-4bb5-ab51-ffd1466268e9 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:51.959114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37be04-3d6c-4bb5-ab51-ffd1466268e9 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:51.964084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22397977-e062-447c-b23a-c799efbe97cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:51.964760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397977-e062-447c-b23a-c799efbe97cf admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:51.965044 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397977-e062-447c-b23a-c799efbe97cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:51.965321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397977-e062-447c-b23a-c799efbe97cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:51.966063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22397977-e062-447c-b23a-c799efbe97cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:14:51.966407 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22397977-e062-447c-b23a-c799efbe97cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:51.968051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22397977-e062-447c-b23a-c799efbe97cf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:51.985196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1293/2599] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:14:51.990239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343e456-d154-4613-99c9-f80e3eed8d16 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.008667 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343e456-d154-4613-99c9-f80e3eed8d16 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.014347 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a074357-89a3-48cb-a558-262c0817d2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.015014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a074357-89a3-48cb-a558-262c0817d2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.015292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a074357-89a3-48cb-a558-262c0817d2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.015561 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a074357-89a3-48cb-a558-262c0817d2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.016292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a074357-89a3-48cb-a558-262c0817d2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.018849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a074357-89a3-48cb-a558-262c0817d2a4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.047488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1307/2600] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:14:51 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:14:52.052077 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22397977-e062-447c-b23a-c799efbe97cf admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.062886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22397977-e062-447c-b23a-c799efbe97cf admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.123230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c534dc-08fd-40de-999f-9a90610e49c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.123914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c534dc-08fd-40de-999f-9a90610e49c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.124197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c534dc-08fd-40de-999f-9a90610e49c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.124471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c534dc-08fd-40de-999f-9a90610e49c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.125183 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c534dc-08fd-40de-999f-9a90610e49c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.127409 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c534dc-08fd-40de-999f-9a90610e49c2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.151618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1294/2601] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:14:52 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:14:52.155846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a074357-89a3-48cb-a558-262c0817d2a4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.167069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a074357-89a3-48cb-a558-262c0817d2a4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.172680 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ca394-b4a3-4782-99ef-f2297cdbebbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.173334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ca394-b4a3-4782-99ef-f2297cdbebbb admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.173618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ca394-b4a3-4782-99ef-f2297cdbebbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.173896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ca394-b4a3-4782-99ef-f2297cdbebbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.174684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ca394-b4a3-4782-99ef-f2297cdbebbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.176339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ca394-b4a3-4782-99ef-f2297cdbebbb admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.187910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1308/2602] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:14:52.192398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c534dc-08fd-40de-999f-9a90610e49c2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.202864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c534dc-08fd-40de-999f-9a90610e49c2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.207935 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769eebe-31a6-4fbb-8875-3c0d36374ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.208614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769eebe-31a6-4fbb-8875-3c0d36374ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.208897 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769eebe-31a6-4fbb-8875-3c0d36374ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.209142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769eebe-31a6-4fbb-8875-3c0d36374ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.209915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769eebe-31a6-4fbb-8875-3c0d36374ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.211576 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769eebe-31a6-4fbb-8875-3c0d36374ed8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.222137 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1295/2603] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:14:52.226434 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ca394-b4a3-4782-99ef-f2297cdbebbb admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.237062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ca394-b4a3-4782-99ef-f2297cdbebbb admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.242468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecfcc1-c336-4dec-a6b5-9f98caf48565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.243372 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecfcc1-c336-4dec-a6b5-9f98caf48565 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.243657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecfcc1-c336-4dec-a6b5-9f98caf48565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.243963 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecfcc1-c336-4dec-a6b5-9f98caf48565 admin admin] PATH_INFO: `/v3/projects/b24a214d021242ec8d747964dcf44076/users/6301a7b26c144d5ea22ecf14c94ba2b4/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.260508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecfcc1-c336-4dec-a6b5-9f98caf48565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b24a214d021242ec8d747964dcf44076, user_id=6301a7b26c144d5ea22ecf14c94ba2b4, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.263229 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecfcc1-c336-4dec-a6b5-9f98caf48565 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.281616 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1309/2604] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:14:52 2022] PUT /identity/v3/projects/b24a214d021242ec8d747964dcf44076/users/6301a7b26c144d5ea22ecf14c94ba2b4/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:52.294159 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dd861-92b2-4e93-8ac1-481745a5d57b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.294451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dd861-92b2-4e93-8ac1-481745a5d57b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.294729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65dd861-92b2-4e93-8ac1-481745a5d57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.325903 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e65dd861-92b2-4e93-8ac1-481745a5d57b None None] MFA Rules not processed for user `6301a7b26c144d5ea22ecf14c94ba2b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:14:52.327210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e65dd861-92b2-4e93-8ac1-481745a5d57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:52.403694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1296/2605] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:14:52 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:14:52.409618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecfcc1-c336-4dec-a6b5-9f98caf48565 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.433514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:52.485660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecfcc1-c336-4dec-a6b5-9f98caf48565 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.490420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73e94c-f8ac-4fbb-b017-3228af23ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.491098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73e94c-f8ac-4fbb-b017-3228af23ae39 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.491380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73e94c-f8ac-4fbb-b017-3228af23ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.491653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73e94c-f8ac-4fbb-b017-3228af23ae39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.492344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73e94c-f8ac-4fbb-b017-3228af23ae39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.494403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73e94c-f8ac-4fbb-b017-3228af23ae39 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.536732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1310/2606] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:14:52 2022] POST /identity/v3/projects => generated 363 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:14:52.538327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dd861-92b2-4e93-8ac1-481745a5d57b None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.551048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65dd861-92b2-4e93-8ac1-481745a5d57b None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.573687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:52.635673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570da58a-c0db-476d-8da1-d0e9a6acbfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.636377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570da58a-c0db-476d-8da1-d0e9a6acbfb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.636656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570da58a-c0db-476d-8da1-d0e9a6acbfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.636927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570da58a-c0db-476d-8da1-d0e9a6acbfb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.637648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570da58a-c0db-476d-8da1-d0e9a6acbfb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.639760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570da58a-c0db-476d-8da1-d0e9a6acbfb3 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.668502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1297/2607] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:14:52 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:14:52.674217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73e94c-f8ac-4fbb-b017-3228af23ae39 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.687207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73e94c-f8ac-4fbb-b017-3228af23ae39 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.692466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81e904-2f68-49ea-9dee-9b2c373aa982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.693077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81e904-2f68-49ea-9dee-9b2c373aa982 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.693350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81e904-2f68-49ea-9dee-9b2c373aa982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.693626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81e904-2f68-49ea-9dee-9b2c373aa982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.694384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81e904-2f68-49ea-9dee-9b2c373aa982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.695992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81e904-2f68-49ea-9dee-9b2c373aa982 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.707709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1311/2608] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:14:52.712143 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570da58a-c0db-476d-8da1-d0e9a6acbfb3 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.722618 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570da58a-c0db-476d-8da1-d0e9a6acbfb3 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.727217 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59358bb6-554f-4926-b46d-e83a6b6ffbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.728115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59358bb6-554f-4926-b46d-e83a6b6ffbf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.728397 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59358bb6-554f-4926-b46d-e83a6b6ffbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.728675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59358bb6-554f-4926-b46d-e83a6b6ffbf1 admin admin] PATH_INFO: `/v3/projects/55375440157144c0955a995a2cd1e6e3/users/5628fa15548b418ea472a515b7d74bc2/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.744739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59358bb6-554f-4926-b46d-e83a6b6ffbf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55375440157144c0955a995a2cd1e6e3, user_id=5628fa15548b418ea472a515b7d74bc2, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.747388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59358bb6-554f-4926-b46d-e83a6b6ffbf1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.761753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1298/2609] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:14:52 2022] PUT /identity/v3/projects/55375440157144c0955a995a2cd1e6e3/users/5628fa15548b418ea472a515b7d74bc2/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:14:52.766435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81e904-2f68-49ea-9dee-9b2c373aa982 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.777530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81e904-2f68-49ea-9dee-9b2c373aa982 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.841756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e670f94-a02f-4abb-9d20-7434c13418c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.842827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e670f94-a02f-4abb-9d20-7434c13418c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.842827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e670f94-a02f-4abb-9d20-7434c13418c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.843062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e670f94-a02f-4abb-9d20-7434c13418c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.843752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e670f94-a02f-4abb-9d20-7434c13418c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.845515 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e670f94-a02f-4abb-9d20-7434c13418c8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.856800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1312/2610] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:14:52.861548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59358bb6-554f-4926-b46d-e83a6b6ffbf1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:52.872877 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59358bb6-554f-4926-b46d-e83a6b6ffbf1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:52.879720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895ec57-71fe-46ae-872b-77ed195976a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:52.880627 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895ec57-71fe-46ae-872b-77ed195976a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.880938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895ec57-71fe-46ae-872b-77ed195976a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.881210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895ec57-71fe-46ae-872b-77ed195976a1 admin admin] PATH_INFO: `/v3/projects/55375440157144c0955a995a2cd1e6e3/users/5628fa15548b418ea472a515b7d74bc2/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.883496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895ec57-71fe-46ae-872b-77ed195976a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55375440157144c0955a995a2cd1e6e3, user_id=5628fa15548b418ea472a515b7d74bc2, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:52.886210 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895ec57-71fe-46ae-872b-77ed195976a1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:52.897779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1299/2611] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:14:52 2022] PUT /identity/v3/projects/55375440157144c0955a995a2cd1e6e3/users/5628fa15548b418ea472a515b7d74bc2/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:14:52.909949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1beae-260d-41b8-b34d-5fe23befe145 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:52.910280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1beae-260d-41b8-b34d-5fe23befe145 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:52.910589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1beae-260d-41b8-b34d-5fe23befe145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:52.940015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-87a1beae-260d-41b8-b34d-5fe23befe145 None None] MFA Rules not processed for user `5628fa15548b418ea472a515b7d74bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:14:52.941239 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-87a1beae-260d-41b8-b34d-5fe23befe145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:53.020756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1313/2612] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:14:52 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:14:53.032786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895ec57-71fe-46ae-872b-77ed195976a1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:53.034822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b895ec57-71fe-46ae-872b-77ed195976a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:53.097477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895ec57-71fe-46ae-872b-77ed195976a1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:53.101612 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc5fc0-702e-410f-9668-b8a04f1e8b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:53.102727 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc5fc0-702e-410f-9668-b8a04f1e8b71 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:53.102827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc5fc0-702e-410f-9668-b8a04f1e8b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:53.103179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc5fc0-702e-410f-9668-b8a04f1e8b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:53.121396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc5fc0-702e-410f-9668-b8a04f1e8b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:53.123041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc5fc0-702e-410f-9668-b8a04f1e8b71 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:53.133931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1300/2613] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:14:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:14:53.211030 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1beae-260d-41b8-b34d-5fe23befe145 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:53.221276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1beae-260d-41b8-b34d-5fe23befe145 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:53.225018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27894048-fc7f-4818-a0aa-98ad71303f2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:53.225668 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27894048-fc7f-4818-a0aa-98ad71303f2f service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:53.225951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27894048-fc7f-4818-a0aa-98ad71303f2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:53.226263 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27894048-fc7f-4818-a0aa-98ad71303f2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:53.238622 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:14:53.292645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27894048-fc7f-4818-a0aa-98ad71303f2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:53.294131 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27894048-fc7f-4818-a0aa-98ad71303f2f service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:53.303711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1314/2614] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:14:53 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:14:54.104203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7644b2-80ce-4dd8-811a-bea6465e80b1 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:54.104706 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7644b2-80ce-4dd8-811a-bea6465e80b1 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:54.104814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7644b2-80ce-4dd8-811a-bea6465e80b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:54.125245 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0c7644b2-80ce-4dd8-811a-bea6465e80b1 None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:14:54.127428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0c7644b2-80ce-4dd8-811a-bea6465e80b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:54.193207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1301/2615] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:14:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:14:54.241236 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27894048-fc7f-4818-a0aa-98ad71303f2f service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:14:54.242816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-27894048-fc7f-4818-a0aa-98ad71303f2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:14:54.315159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27894048-fc7f-4818-a0aa-98ad71303f2f service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:14:54.321405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbfce1-253d-4f40-a84b-a03304866eb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:14:54.322075 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbfce1-253d-4f40-a84b-a03304866eb7 service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:14:54.322492 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbfce1-253d-4f40-a84b-a03304866eb7 service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:14:54.322871 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbfce1-253d-4f40-a84b-a03304866eb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:14:54.333640 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbfce1-253d-4f40-a84b-a03304866eb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:14:54.335295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbfce1-253d-4f40-a84b-a03304866eb7 service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:14:54.344077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1315/2616] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:14:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:15:04.185416 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:04.340890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=53 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:06.315919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7644b2-80ce-4dd8-811a-bea6465e80b1 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:06.389651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7644b2-80ce-4dd8-811a-bea6465e80b1 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:06.395383 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:06.396295 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:06.396577 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:06.396850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] PATH_INFO: `/v3/users/6301a7b26c144d5ea22ecf14c94ba2b4` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:06.398153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6301a7b26c144d5ea22ecf14c94ba2b4)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:06.400152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:06.463845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6301a7b26c144d5ea22ecf14c94ba2b4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:06.464528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6301a7b26c144d5ea22ecf14c94ba2b4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:06.474081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6301a7b26c144d5ea22ecf14c94ba2b4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:06.475068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6301a7b26c144d5ea22ecf14c94ba2b4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:06.487168 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6301a7b26c144d5ea22ecf14c94ba2b4'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:06.513995 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1302/2617] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:15:06 2022] DELETE /identity/v3/users/6301a7b26c144d5ea22ecf14c94ba2b4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:06.519496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2531d-0a86-4776-b6ac-577a15ad2efc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:06.520132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2531d-0a86-4776-b6ac-577a15ad2efc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:06.520132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2531d-0a86-4776-b6ac-577a15ad2efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:06.539054 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-49f2531d-0a86-4776-b6ac-577a15ad2efc None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:06.540287 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-49f2531d-0a86-4776-b6ac-577a15ad2efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:06.607845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1316/2618] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:06.617816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:06.619555 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:06.677979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8e7a4-b2d8-43f1-8ea0-4f9e73b674b4 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:06.682760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8f80e-11d7-4b00-811e-c837f86e64c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:06.683569 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8f80e-11d7-4b00-811e-c837f86e64c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:06.684438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8f80e-11d7-4b00-811e-c837f86e64c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:06.684735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8f80e-11d7-4b00-811e-c837f86e64c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:06.696869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8f80e-11d7-4b00-811e-c837f86e64c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:06.698471 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8f80e-11d7-4b00-811e-c837f86e64c0 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:06.709896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1303/2619] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:15:07.014490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2531d-0a86-4776-b6ac-577a15ad2efc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:07.016175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-49f2531d-0a86-4776-b6ac-577a15ad2efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:07.035506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2531d-0a86-4776-b6ac-577a15ad2efc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:07.043708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:07.045000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:07.045287 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:07.045567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] PATH_INFO: `/v3/projects/b24a214d021242ec8d747964dcf44076` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:07.047072 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b24a214d021242ec8d747964dcf44076)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:07.049194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:07.115546 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b24a214d021242ec8d747964dcf44076', 'request_id': 'req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.125161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b24a214d021242ec8d747964dcf44076', 'request_id': 'req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.135257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b24a214d021242ec8d747964dcf44076', 'request_id': 'req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.151215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] The token cache is being invalidate because project b24a214d021242ec8d747964dcf44076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:15:07.151610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.152435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.154745 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1317/2620] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:15:07 2022] DELETE /identity/v3/projects/b24a214d021242ec8d747964dcf44076 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:15:07.160132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8f80e-11d7-4b00-811e-c837f86e64c0 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:07.173834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8f80e-11d7-4b00-811e-c837f86e64c0 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:07.203432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:07.258514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:07.259436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:07.259728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:07.260011 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] PATH_INFO: `/v3/users/5628fa15548b418ea472a515b7d74bc2` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:07.261332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5628fa15548b418ea472a515b7d74bc2)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:07.263344 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:07.334042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5628fa15548b418ea472a515b7d74bc2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.335058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5628fa15548b418ea472a515b7d74bc2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.346581 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5628fa15548b418ea472a515b7d74bc2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.347511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5628fa15548b418ea472a515b7d74bc2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.358728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b92fa04a-2bdc-4115-abe3-322b9a9bf251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5628fa15548b418ea472a515b7d74bc2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.387898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1304/2621] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:15:07 2022] DELETE /identity/v3/users/5628fa15548b418ea472a515b7d74bc2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:07.752639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:07.755024 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:07.783095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe0c24-b27f-44f7-b1f0-f8d6061ee8ae admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:07.870577 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:07.871495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:07.871734 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:07.872015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] PATH_INFO: `/v3/projects/55375440157144c0955a995a2cd1e6e3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:07.873230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=55375440157144c0955a995a2cd1e6e3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:07.875317 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:07.920642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:07.941670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55375440157144c0955a995a2cd1e6e3', 'request_id': 'req-e580ec87-4234-4af8-af70-357b9843b9ad'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.950119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55375440157144c0955a995a2cd1e6e3', 'request_id': 'req-e580ec87-4234-4af8-af70-357b9843b9ad'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.956503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55375440157144c0955a995a2cd1e6e3', 'request_id': 'req-e580ec87-4234-4af8-af70-357b9843b9ad'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.968266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] The token cache is being invalidate because project 55375440157144c0955a995a2cd1e6e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:15:07.968554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.969391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e580ec87-4234-4af8-af70-357b9843b9ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:07.971152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1318/2622] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:15:07 2022] DELETE /identity/v3/projects/55375440157144c0955a995a2cd1e6e3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:07.976401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d96d3-1696-44ee-abc5-c44fa8984828 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:07.976687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d96d3-1696-44ee-abc5-c44fa8984828 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:07.976967 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d96d3-1696-44ee-abc5-c44fa8984828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:07.995357 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-146d96d3-1696-44ee-abc5-c44fa8984828 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:07.996578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-146d96d3-1696-44ee-abc5-c44fa8984828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:08.071127 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1305/2623] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:08.083205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125434f-7ca6-4eb4-9ce8-068e384fd194 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.084208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125434f-7ca6-4eb4-9ce8-068e384fd194 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.084700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125434f-7ca6-4eb4-9ce8-068e384fd194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.115013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4678912-360b-48f2-90d1-5bcd6639ebb8 None None] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.115396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4678912-360b-48f2-90d1-5bcd6639ebb8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.115747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4678912-360b-48f2-90d1-5bcd6639ebb8 None None] PATH_INFO: `/` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.117430 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1306/2624] 10.222.0.33 () {54 vars in 921 bytes} [Mon Jun 20 11:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 11:15:08.119414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d125434f-7ca6-4eb4-9ce8-068e384fd194 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:08.121451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d125434f-7ca6-4eb4-9ce8-068e384fd194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:08.121932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ea366-c674-4ed5-ab09-048456e71c6c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.122263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ea366-c674-4ed5-ab09-048456e71c6c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.122619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ea366-c674-4ed5-ab09-048456e71c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.136482 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1319/2625] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:08.140409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125434f-7ca6-4eb4-9ce8-068e384fd194 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:08.143910 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-657ea366-c674-4ed5-ab09-048456e71c6c None None] MFA Rules not processed for user `37b1f13aa474493d83b3a16f76949fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:08.148846 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-657ea366-c674-4ed5-ab09-048456e71c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:08.150647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125434f-7ca6-4eb4-9ce8-068e384fd194 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:08.155891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5f349-c6cf-425d-b21a-966d04121355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:08.156679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5f349-c6cf-425d-b21a-966d04121355 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.157117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5f349-c6cf-425d-b21a-966d04121355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.157976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5f349-c6cf-425d-b21a-966d04121355 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.158918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5f349-c6cf-425d-b21a-966d04121355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:15:08.159228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5f349-c6cf-425d-b21a-966d04121355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:08.161035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5f349-c6cf-425d-b21a-966d04121355 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:08.174134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1320/2626] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:15:08.178181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5f349-c6cf-425d-b21a-966d04121355 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:08.191248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5f349-c6cf-425d-b21a-966d04121355 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:08.197154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccf45b-dba4-4ba5-8403-392635e88eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:08.198078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccf45b-dba4-4ba5-8403-392635e88eea admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.198297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccf45b-dba4-4ba5-8403-392635e88eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.198438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccf45b-dba4-4ba5-8403-392635e88eea admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.199125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccf45b-dba4-4ba5-8403-392635e88eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:08.201071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccf45b-dba4-4ba5-8403-392635e88eea admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:08.223274 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:08.232693 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1321/2627] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:15:08 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:15:08.237791 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccf45b-dba4-4ba5-8403-392635e88eea admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:08.271232 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1307/2628] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Jun 20 11:15:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:15:08.286304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ea366-c674-4ed5-ab09-048456e71c6c None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:08.287852 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-657ea366-c674-4ed5-ab09-048456e71c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:08.326344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccf45b-dba4-4ba5-8403-392635e88eea admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:08.335647 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae3113-c579-48fe-ad1d-cd8cc1f4e8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:08.337535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae3113-c579-48fe-ad1d-cd8cc1f4e8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.337997 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae3113-c579-48fe-ad1d-cd8cc1f4e8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.338321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae3113-c579-48fe-ad1d-cd8cc1f4e8e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.339061 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae3113-c579-48fe-ad1d-cd8cc1f4e8e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:08.341572 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae3113-c579-48fe-ad1d-cd8cc1f4e8e7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:08.362505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ea366-c674-4ed5-ab09-048456e71c6c None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:08.372952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1322/2629] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:15:08 2022] POST /identity/v3/users => generated 286 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:15:08.379740 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae3113-c579-48fe-ad1d-cd8cc1f4e8e7 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:08.391058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae3113-c579-48fe-ad1d-cd8cc1f4e8e7 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:08.396973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a844a4-6b23-4ef5-9b67-ec275b436843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:08.397846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a844a4-6b23-4ef5-9b67-ec275b436843 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.397976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a844a4-6b23-4ef5-9b67-ec275b436843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.398269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a844a4-6b23-4ef5-9b67-ec275b436843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.399327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a844a4-6b23-4ef5-9b67-ec275b436843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:08.401005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a844a4-6b23-4ef5-9b67-ec275b436843 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:08.413827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1323/2630] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:08.417490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a844a4-6b23-4ef5-9b67-ec275b436843 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:08.437610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a844a4-6b23-4ef5-9b67-ec275b436843 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:08.442716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acc3e7-a633-40ed-9355-2d62d1ec76e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4bf4d9571d541bcbba87094198dc2ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:08.443728 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb1e67-0b20-429a-bb02-4928b90c316c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:08.443871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acc3e7-a633-40ed-9355-2d62d1ec76e1 service glance] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.444201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acc3e7-a633-40ed-9355-2d62d1ec76e1 service glance] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.444478 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85acc3e7-a633-40ed-9355-2d62d1ec76e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.445491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb1e67-0b20-429a-bb02-4928b90c316c admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.445953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb1e67-0b20-429a-bb02-4928b90c316c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.446420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb1e67-0b20-429a-bb02-4928b90c316c admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.447348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb1e67-0b20-429a-bb02-4928b90c316c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:08.449316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb1e67-0b20-429a-bb02-4928b90c316c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:08.462031 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1324/2631] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:08.466310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb1e67-0b20-429a-bb02-4928b90c316c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:08.480716 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb1e67-0b20-429a-bb02-4928b90c316c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:08.482591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169cca8-6526-4e87-baef-c5d7c28e0563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:08.483547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169cca8-6526-4e87-baef-c5d7c28e0563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.484000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169cca8-6526-4e87-baef-c5d7c28e0563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.484280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169cca8-6526-4e87-baef-c5d7c28e0563 admin admin] PATH_INFO: `/v3/projects/4331b015d952405282756ae8be2af9ec/users/5ab083a386344209b8dd5ed35e40b877/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.503375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169cca8-6526-4e87-baef-c5d7c28e0563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4331b015d952405282756ae8be2af9ec, user_id=5ab083a386344209b8dd5ed35e40b877, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:08.506173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169cca8-6526-4e87-baef-c5d7c28e0563 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:08.525344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1325/2632] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:15:08 2022] PUT /identity/v3/projects/4331b015d952405282756ae8be2af9ec/users/5ab083a386344209b8dd5ed35e40b877/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:15:08.533943 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab45011-db08-460d-b714-7e80bdf90ce7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.534269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab45011-db08-460d-b714-7e80bdf90ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.534708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab45011-db08-460d-b714-7e80bdf90ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.551560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acc3e7-a633-40ed-9355-2d62d1ec76e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:08.553132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85acc3e7-a633-40ed-9355-2d62d1ec76e1 service glance] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:08.566571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cab45011-db08-460d-b714-7e80bdf90ce7 None None] MFA Rules not processed for user `5ab083a386344209b8dd5ed35e40b877`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:08.568073 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cab45011-db08-460d-b714-7e80bdf90ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:08.569236 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1308/2633] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:15:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:15:08.573044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:08.666499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1326/2634] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:15:08.668097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acc3e7-a633-40ed-9355-2d62d1ec76e1 service glance] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:08.685938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85acc3e7-a633-40ed-9355-2d62d1ec76e1 service glance] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:08.726622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:08.772759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858142b-5761-4ffc-9cc8-79ab2761332e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:08.774106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858142b-5761-4ffc-9cc8-79ab2761332e admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.774106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858142b-5761-4ffc-9cc8-79ab2761332e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.774391 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858142b-5761-4ffc-9cc8-79ab2761332e admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.775069 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858142b-5761-4ffc-9cc8-79ab2761332e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:08.777028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858142b-5761-4ffc-9cc8-79ab2761332e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:08.824142 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1309/2635] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:15:08 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:15:08.825543 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab45011-db08-460d-b714-7e80bdf90ce7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:08.837381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab45011-db08-460d-b714-7e80bdf90ce7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:08.915001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4985c3-0ef9-447b-89f7-d80d19837808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:08.915688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4985c3-0ef9-447b-89f7-d80d19837808 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.915965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4985c3-0ef9-447b-89f7-d80d19837808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.916235 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4985c3-0ef9-447b-89f7-d80d19837808 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.916927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4985c3-0ef9-447b-89f7-d80d19837808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:08.919175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4985c3-0ef9-447b-89f7-d80d19837808 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:08.942132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1327/2636] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:15:08 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:15:08.946349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858142b-5761-4ffc-9cc8-79ab2761332e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:08.959088 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858142b-5761-4ffc-9cc8-79ab2761332e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:08.963930 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67c2d6-1cad-4923-9706-a82fa8b9f866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:08.964614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67c2d6-1cad-4923-9706-a82fa8b9f866 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:08.964914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67c2d6-1cad-4923-9706-a82fa8b9f866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:08.965185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67c2d6-1cad-4923-9706-a82fa8b9f866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:08.965919 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67c2d6-1cad-4923-9706-a82fa8b9f866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:08.967571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67c2d6-1cad-4923-9706-a82fa8b9f866 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:08.982418 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1310/2637] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:15:08.996214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4985c3-0ef9-447b-89f7-d80d19837808 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:09.007029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4985c3-0ef9-447b-89f7-d80d19837808 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:09.012889 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835da08d-fd71-406b-af85-068145b6f89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:09.013919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835da08d-fd71-406b-af85-068145b6f89b admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:09.014605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835da08d-fd71-406b-af85-068145b6f89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:09.014605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835da08d-fd71-406b-af85-068145b6f89b admin admin] PATH_INFO: `/v3/projects/abb2d39c10cb44f2872873fbc90d60b1/users/f3328a0092ae49b4bec7e009e167f5ac/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:09.035646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835da08d-fd71-406b-af85-068145b6f89b admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb2d39c10cb44f2872873fbc90d60b1, user_id=f3328a0092ae49b4bec7e009e167f5ac, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:09.038407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835da08d-fd71-406b-af85-068145b6f89b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:09.063678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1328/2638] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:15:08 2022] PUT /identity/v3/projects/abb2d39c10cb44f2872873fbc90d60b1/users/f3328a0092ae49b4bec7e009e167f5ac/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:09.068304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67c2d6-1cad-4923-9706-a82fa8b9f866 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:09.079388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67c2d6-1cad-4923-9706-a82fa8b9f866 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:09.154188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f44a37-02ee-42a8-8502-8317cdfe4886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:09.155150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f44a37-02ee-42a8-8502-8317cdfe4886 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:09.155326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f44a37-02ee-42a8-8502-8317cdfe4886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:09.155604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f44a37-02ee-42a8-8502-8317cdfe4886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:09.156316 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f44a37-02ee-42a8-8502-8317cdfe4886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:09.158031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f44a37-02ee-42a8-8502-8317cdfe4886 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:09.173832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1311/2639] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:09.177723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835da08d-fd71-406b-af85-068145b6f89b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:09.190535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835da08d-fd71-406b-af85-068145b6f89b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:09.195382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7c39-d77a-4630-8bda-d3fd64e471ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:09.196296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7c39-d77a-4630-8bda-d3fd64e471ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:09.196575 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7c39-d77a-4630-8bda-d3fd64e471ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:09.196847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7c39-d77a-4630-8bda-d3fd64e471ba admin admin] PATH_INFO: `/v3/projects/abb2d39c10cb44f2872873fbc90d60b1/users/f3328a0092ae49b4bec7e009e167f5ac/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:09.198822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb7c39-d77a-4630-8bda-d3fd64e471ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb2d39c10cb44f2872873fbc90d60b1, user_id=f3328a0092ae49b4bec7e009e167f5ac, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:09.201449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb7c39-d77a-4630-8bda-d3fd64e471ba admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:09.217300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1329/2640] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:15:09 2022] PUT /identity/v3/projects/abb2d39c10cb44f2872873fbc90d60b1/users/f3328a0092ae49b4bec7e009e167f5ac/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:09.227655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c700a-5c7f-4021-9a44-31ce1674fed0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:09.228202 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c700a-5c7f-4021-9a44-31ce1674fed0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:09.228496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c700a-5c7f-4021-9a44-31ce1674fed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:09.262317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-638c700a-5c7f-4021-9a44-31ce1674fed0 None None] MFA Rules not processed for user `f3328a0092ae49b4bec7e009e167f5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:09.263704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-638c700a-5c7f-4021-9a44-31ce1674fed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:09.336296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1312/2641] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:09.348074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7c39-d77a-4630-8bda-d3fd64e471ba admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:09.350275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-47cb7c39-d77a-4630-8bda-d3fd64e471ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:09.417288 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7c39-d77a-4630-8bda-d3fd64e471ba admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:09.422950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e83ad9-e814-472b-97cf-69ea534d1423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:09.423628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e83ad9-e814-472b-97cf-69ea534d1423 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:09.423918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e83ad9-e814-472b-97cf-69ea534d1423 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:09.424199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e83ad9-e814-472b-97cf-69ea534d1423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:09.434912 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:09.488730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e83ad9-e814-472b-97cf-69ea534d1423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:09.490222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e83ad9-e814-472b-97cf-69ea534d1423 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:09.500407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1330/2642] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:15:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:15:11.033116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16678374-4da1-4135-aca9-b11b5bb8adc3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:11.034299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16678374-4da1-4135-aca9-b11b5bb8adc3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:11.034976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16678374-4da1-4135-aca9-b11b5bb8adc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:11.055476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-16678374-4da1-4135-aca9-b11b5bb8adc3 None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:11.056662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-16678374-4da1-4135-aca9-b11b5bb8adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:11.083015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:11.127371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1313/2643] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:15:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:11.143560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e83ad9-e814-472b-97cf-69ea534d1423 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:11.145396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-64e83ad9-e814-472b-97cf-69ea534d1423 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:11.247018 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e83ad9-e814-472b-97cf-69ea534d1423 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:11.252634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92a18d-2106-47ea-8e3f-71ab4cf73a4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:11.253306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92a18d-2106-47ea-8e3f-71ab4cf73a4f service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:11.253595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92a18d-2106-47ea-8e3f-71ab4cf73a4f service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:11.253911 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92a18d-2106-47ea-8e3f-71ab4cf73a4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:11.264343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92a18d-2106-47ea-8e3f-71ab4cf73a4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:11.266119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92a18d-2106-47ea-8e3f-71ab4cf73a4f service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:11.276143 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1331/2644] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:15:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:15:19.498163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16678374-4da1-4135-aca9-b11b5bb8adc3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:19.500614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-16678374-4da1-4135-aca9-b11b5bb8adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:19.514998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16678374-4da1-4135-aca9-b11b5bb8adc3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:19.519938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6fcea-58d9-4570-a8aa-65f51c56de58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:19.520626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6fcea-58d9-4570-a8aa-65f51c56de58 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:19.520883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6fcea-58d9-4570-a8aa-65f51c56de58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:19.521172 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6fcea-58d9-4570-a8aa-65f51c56de58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:19.532781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6fcea-58d9-4570-a8aa-65f51c56de58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:19.534522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6fcea-58d9-4570-a8aa-65f51c56de58 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:19.546150 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1314/2645] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:15:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:15:21.272184 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=54 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:29.544528 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=19 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:49.550863 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92a18d-2106-47ea-8e3f-71ab4cf73a4f service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:49.552593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2b92a18d-2106-47ea-8e3f-71ab4cf73a4f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:49.657593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92a18d-2106-47ea-8e3f-71ab4cf73a4f service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:49.663605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c529f-08c7-4531-8747-f5c5651444ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:49.664271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c529f-08c7-4531-8747-f5c5651444ef service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:49.664598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c529f-08c7-4531-8747-f5c5651444ef service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:49.664895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c529f-08c7-4531-8747-f5c5651444ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:49.666910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4a8c529f-08c7-4531-8747-f5c5651444ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:49.679998 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c529f-08c7-4531-8747-f5c5651444ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:49.681522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c529f-08c7-4531-8747-f5c5651444ef service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:49.693326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1332/2646] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:15:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:15:50.296889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6fcea-58d9-4570-a8aa-65f51c56de58 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:50.298609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-20e6fcea-58d9-4570-a8aa-65f51c56de58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:50.385622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6fcea-58d9-4570-a8aa-65f51c56de58 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:50.393786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cf30c-b880-4c0e-b1e9-ee768a45331a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:50.394549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cf30c-b880-4c0e-b1e9-ee768a45331a service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:50.394829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cf30c-b880-4c0e-b1e9-ee768a45331a service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:50.395103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cf30c-b880-4c0e-b1e9-ee768a45331a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:50.397035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fd3cf30c-b880-4c0e-b1e9-ee768a45331a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:50.408730 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cf30c-b880-4c0e-b1e9-ee768a45331a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:50.410109 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cf30c-b880-4c0e-b1e9-ee768a45331a service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:50.421992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1315/2647] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:15:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:15:52.556960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea631ce-587c-4068-a133-a82724e97c56 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:52.558820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea631ce-587c-4068-a133-a82724e97c56 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:52.559249 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea631ce-587c-4068-a133-a82724e97c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:52.577678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0ea631ce-587c-4068-a133-a82724e97c56 None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:52.578909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0ea631ce-587c-4068-a133-a82724e97c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:52.586407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1333/2648] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jun 20 11:15:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:52.599625 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cf30c-b880-4c0e-b1e9-ee768a45331a service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:52.612684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cf30c-b880-4c0e-b1e9-ee768a45331a service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:52.617866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a09a4d-a2e8-485b-abc3-53cb54114bb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:52.618535 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a09a4d-a2e8-485b-abc3-53cb54114bb7 service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:52.618818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a09a4d-a2e8-485b-abc3-53cb54114bb7 service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:52.619097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a09a4d-a2e8-485b-abc3-53cb54114bb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:52.628673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a09a4d-a2e8-485b-abc3-53cb54114bb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:52.630339 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a09a4d-a2e8-485b-abc3-53cb54114bb7 service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:52.640512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1316/2649] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:15:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:15:53.016885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea631ce-587c-4068-a133-a82724e97c56 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:53.086452 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea631ce-587c-4068-a133-a82724e97c56 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:53.091304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:53.092206 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:53.092493 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:53.092765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] PATH_INFO: `/v3/users/5ab083a386344209b8dd5ed35e40b877` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:53.093955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ab083a386344209b8dd5ed35e40b877)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:53.095924 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:53.152445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ab083a386344209b8dd5ed35e40b877'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.163777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ab083a386344209b8dd5ed35e40b877'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.164725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ab083a386344209b8dd5ed35e40b877'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.165340 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ab083a386344209b8dd5ed35e40b877'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.181505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ab083a386344209b8dd5ed35e40b877'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.191331 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:53.197823 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1334/2650] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:15:53 2022] DELETE /identity/v3/users/5ab083a386344209b8dd5ed35e40b877 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:53.208606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e2e1f-b820-47dc-a719-6db5f1f6d140 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:53.209135 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e2e1f-b820-47dc-a719-6db5f1f6d140 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:53.209606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e2e1f-b820-47dc-a719-6db5f1f6d140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:53.229900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f50e2e1f-b820-47dc-a719-6db5f1f6d140 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:53.232609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f50e2e1f-b820-47dc-a719-6db5f1f6d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:53.304534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1317/2651] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:53.315067 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:53.316579 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:53.378203 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1bcff-8c85-4e90-850f-1d75e46560bb admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:53.382428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a877d9-0291-4110-9e0e-59ecc0797a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:53.383111 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a877d9-0291-4110-9e0e-59ecc0797a41 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:53.383382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a877d9-0291-4110-9e0e-59ecc0797a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:53.383634 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a877d9-0291-4110-9e0e-59ecc0797a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:53.393605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a877d9-0291-4110-9e0e-59ecc0797a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:53.395092 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a877d9-0291-4110-9e0e-59ecc0797a41 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:53.404946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1335/2652] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:15:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:15:53.703667 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e2e1f-b820-47dc-a719-6db5f1f6d140 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:53.705187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f50e2e1f-b820-47dc-a719-6db5f1f6d140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:53.723842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e2e1f-b820-47dc-a719-6db5f1f6d140 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:53.729250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:53.730325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:53.731465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:53.731755 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] PATH_INFO: `/v3/projects/4331b015d952405282756ae8be2af9ec` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:53.733340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4331b015d952405282756ae8be2af9ec)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:53.740436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:53.800875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4331b015d952405282756ae8be2af9ec', 'request_id': 'req-25b3d2d4-e48d-4e54-8902-376a63040629'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.811596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4331b015d952405282756ae8be2af9ec', 'request_id': 'req-25b3d2d4-e48d-4e54-8902-376a63040629'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.818035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4331b015d952405282756ae8be2af9ec', 'request_id': 'req-25b3d2d4-e48d-4e54-8902-376a63040629'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.829739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] The token cache is being invalidate because project 4331b015d952405282756ae8be2af9ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:15:53.830028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.830724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.832306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1318/2653] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:15:53 2022] DELETE /identity/v3/projects/4331b015d952405282756ae8be2af9ec => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:15:53.836321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a877d9-0291-4110-9e0e-59ecc0797a41 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:53.847225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a877d9-0291-4110-9e0e-59ecc0797a41 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:53.910460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:53.911372 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:53.911653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:53.911931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] PATH_INFO: `/v3/users/f3328a0092ae49b4bec7e009e167f5ac` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:53.915540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3328a0092ae49b4bec7e009e167f5ac)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:53.917595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:53.978142 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3328a0092ae49b4bec7e009e167f5ac'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.987290 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3328a0092ae49b4bec7e009e167f5ac'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.987960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3328a0092ae49b4bec7e009e167f5ac'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:53.988586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3328a0092ae49b4bec7e009e167f5ac'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:54.007007 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-01ad3b36-1657-4a14-8937-d3349f29ecd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3328a0092ae49b4bec7e009e167f5ac'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:54.023385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1336/2654] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:15:53 2022] DELETE /identity/v3/users/f3328a0092ae49b4bec7e009e167f5ac => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:54.322625 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:54.325079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:54.348827 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3d2d4-e48d-4e54-8902-376a63040629 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:54.361076 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:54.415649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:54.416547 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:54.416833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:54.417114 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] PATH_INFO: `/v3/projects/abb2d39c10cb44f2872873fbc90d60b1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:54.418601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb2d39c10cb44f2872873fbc90d60b1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:54.420546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:54.475029 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb2d39c10cb44f2872873fbc90d60b1', 'request_id': 'req-b9c0a1fa-0e37-49df-844e-bf719e105c0d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:54.483648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb2d39c10cb44f2872873fbc90d60b1', 'request_id': 'req-b9c0a1fa-0e37-49df-844e-bf719e105c0d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:54.489570 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb2d39c10cb44f2872873fbc90d60b1', 'request_id': 'req-b9c0a1fa-0e37-49df-844e-bf719e105c0d'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:54.501476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] The token cache is being invalidate because project abb2d39c10cb44f2872873fbc90d60b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:15:54.501766 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:54.502525 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-b9c0a1fa-0e37-49df-844e-bf719e105c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:54.504123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1319/2655] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:15:54 2022] DELETE /identity/v3/projects/abb2d39c10cb44f2872873fbc90d60b1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:54.509468 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80759a3-b026-4da2-bb5f-e506cd7e27fd None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:54.509754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80759a3-b026-4da2-bb5f-e506cd7e27fd None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:54.510035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80759a3-b026-4da2-bb5f-e506cd7e27fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:54.532446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a80759a3-b026-4da2-bb5f-e506cd7e27fd None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:54.533676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a80759a3-b026-4da2-bb5f-e506cd7e27fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:54.549064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:54.610466 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1337/2656] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:54.618647 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226227de-7394-4b24-8174-cf4d333ed1d0 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:54.618928 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226227de-7394-4b24-8174-cf4d333ed1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:54.619196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226227de-7394-4b24-8174-cf4d333ed1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:54.640126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-226227de-7394-4b24-8174-cf4d333ed1d0 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:54.641343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-226227de-7394-4b24-8174-cf4d333ed1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:54.649602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1320/2657] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:54.654183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80759a3-b026-4da2-bb5f-e506cd7e27fd None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:54.664140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80759a3-b026-4da2-bb5f-e506cd7e27fd None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:54.669431 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41b03-a2eb-4da3-8f18-7de73631a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:54.670129 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41b03-a2eb-4da3-8f18-7de73631a97f admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:54.670412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41b03-a2eb-4da3-8f18-7de73631a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:54.670687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d41b03-a2eb-4da3-8f18-7de73631a97f admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:54.671416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d41b03-a2eb-4da3-8f18-7de73631a97f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:15:54.671718 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d41b03-a2eb-4da3-8f18-7de73631a97f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:54.673343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d41b03-a2eb-4da3-8f18-7de73631a97f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:54.687124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1338/2658] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:15:54.691721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226227de-7394-4b24-8174-cf4d333ed1d0 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:54.702261 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226227de-7394-4b24-8174-cf4d333ed1d0 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:54.707592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69007f-e987-4b4d-8881-75a3f773816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:54.708295 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69007f-e987-4b4d-8881-75a3f773816f admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:54.708575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69007f-e987-4b4d-8881-75a3f773816f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:54.708849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69007f-e987-4b4d-8881-75a3f773816f admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:54.709548 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69007f-e987-4b4d-8881-75a3f773816f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:54.711545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69007f-e987-4b4d-8881-75a3f773816f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:54.744253 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1321/2659] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:15:54 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:15:54.748666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41b03-a2eb-4da3-8f18-7de73631a97f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:54.759175 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d41b03-a2eb-4da3-8f18-7de73631a97f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:54.821101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaefa5-1a0e-41b6-a053-e6fa88ec3ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:54.821812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaefa5-1a0e-41b6-a053-e6fa88ec3ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:54.822114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaefa5-1a0e-41b6-a053-e6fa88ec3ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:54.822397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaefa5-1a0e-41b6-a053-e6fa88ec3ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:54.823124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaefa5-1a0e-41b6-a053-e6fa88ec3ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:54.825199 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaefa5-1a0e-41b6-a053-e6fa88ec3ca9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:54.851760 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1339/2660] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:15:54 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:15:54.857911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69007f-e987-4b4d-8881-75a3f773816f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:54.869931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69007f-e987-4b4d-8881-75a3f773816f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:54.875756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c006dd-f584-46cf-931c-723f3839f609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:54.876442 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c006dd-f584-46cf-931c-723f3839f609 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:54.876720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c006dd-f584-46cf-931c-723f3839f609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:54.876990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c006dd-f584-46cf-931c-723f3839f609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:54.877728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c006dd-f584-46cf-931c-723f3839f609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:54.879421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c006dd-f584-46cf-931c-723f3839f609 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:54.892769 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1322/2661] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:54.897205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaefa5-1a0e-41b6-a053-e6fa88ec3ca9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:54.908930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaefa5-1a0e-41b6-a053-e6fa88ec3ca9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:54.915657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74a5e5-bd60-465d-a664-cf0a71e55068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:54.916380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74a5e5-bd60-465d-a664-cf0a71e55068 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:54.916664 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74a5e5-bd60-465d-a664-cf0a71e55068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:54.916941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74a5e5-bd60-465d-a664-cf0a71e55068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:54.917876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74a5e5-bd60-465d-a664-cf0a71e55068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:54.920208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74a5e5-bd60-465d-a664-cf0a71e55068 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:54.931403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1340/2662] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:54.936401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c006dd-f584-46cf-931c-723f3839f609 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:54.947128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c006dd-f584-46cf-931c-723f3839f609 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:54.953052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc9e0d-5d8c-41ae-b2b2-bc558517985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:54.953953 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc9e0d-5d8c-41ae-b2b2-bc558517985a admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:54.954263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc9e0d-5d8c-41ae-b2b2-bc558517985a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:54.954535 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc9e0d-5d8c-41ae-b2b2-bc558517985a admin admin] PATH_INFO: `/v3/projects/08c8b792095f4f9a80263ac9ee2dc33a/users/a57c2c2857504b968014752f568e775f/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:54.975911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc9e0d-5d8c-41ae-b2b2-bc558517985a admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c8b792095f4f9a80263ac9ee2dc33a, user_id=a57c2c2857504b968014752f568e775f, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:54.978681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc9e0d-5d8c-41ae-b2b2-bc558517985a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:54.996079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1323/2663] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:15:54 2022] PUT /identity/v3/projects/08c8b792095f4f9a80263ac9ee2dc33a/users/a57c2c2857504b968014752f568e775f/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:55.007509 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096b4a4-7a16-42b7-8e46-981015941d6f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:55.007777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096b4a4-7a16-42b7-8e46-981015941d6f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:55.008084 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096b4a4-7a16-42b7-8e46-981015941d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:55.036274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2096b4a4-7a16-42b7-8e46-981015941d6f None None] MFA Rules not processed for user `a57c2c2857504b968014752f568e775f`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:55.037537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2096b4a4-7a16-42b7-8e46-981015941d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:55.106217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1341/2664] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:55.112915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc9e0d-5d8c-41ae-b2b2-bc558517985a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:55.162408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:55.185716 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc9e0d-5d8c-41ae-b2b2-bc558517985a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:55.191057 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694befe3-28d5-4183-bd19-1b41a1307d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:55.191726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694befe3-28d5-4183-bd19-1b41a1307d99 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:55.192036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694befe3-28d5-4183-bd19-1b41a1307d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:55.192263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694befe3-28d5-4183-bd19-1b41a1307d99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:55.192962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694befe3-28d5-4183-bd19-1b41a1307d99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:55.194906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694befe3-28d5-4183-bd19-1b41a1307d99 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:55.222054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1324/2665] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:15:55 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:15:55.226395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096b4a4-7a16-42b7-8e46-981015941d6f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:55.238277 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096b4a4-7a16-42b7-8e46-981015941d6f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:55.251862 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:55.305260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ccb4b-f9d1-4883-af77-2139c40ab6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:55.305959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ccb4b-f9d1-4883-af77-2139c40ab6dd admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:55.306273 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ccb4b-f9d1-4883-af77-2139c40ab6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:55.306554 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ccb4b-f9d1-4883-af77-2139c40ab6dd admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:55.307268 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ccb4b-f9d1-4883-af77-2139c40ab6dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:55.310005 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ccb4b-f9d1-4883-af77-2139c40ab6dd admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:55.344975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1342/2666] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:15:55 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:15:55.346572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694befe3-28d5-4183-bd19-1b41a1307d99 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:55.358153 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694befe3-28d5-4183-bd19-1b41a1307d99 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:55.363476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521a56f-2932-43cb-acb7-fd9e931d332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:55.364182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521a56f-2932-43cb-acb7-fd9e931d332f admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:55.364452 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521a56f-2932-43cb-acb7-fd9e931d332f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:55.364733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521a56f-2932-43cb-acb7-fd9e931d332f admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:55.365497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521a56f-2932-43cb-acb7-fd9e931d332f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:55.367174 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521a56f-2932-43cb-acb7-fd9e931d332f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:55.380103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1325/2667] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:55.384957 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ccb4b-f9d1-4883-af77-2139c40ab6dd admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:55.396144 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ccb4b-f9d1-4883-af77-2139c40ab6dd admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:55.402169 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e57c4e-19d7-46c2-af6f-9bb1f489e3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:55.403377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e57c4e-19d7-46c2-af6f-9bb1f489e3c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:55.403756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e57c4e-19d7-46c2-af6f-9bb1f489e3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:55.403921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e57c4e-19d7-46c2-af6f-9bb1f489e3c4 admin admin] PATH_INFO: `/v3/projects/6e5646f63528416dadefa845e1c80ee8/users/d8f01735f81f4ef3bc782a60f7c3bc3a/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:55.420807 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e57c4e-19d7-46c2-af6f-9bb1f489e3c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5646f63528416dadefa845e1c80ee8, user_id=d8f01735f81f4ef3bc782a60f7c3bc3a, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:55.423510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e57c4e-19d7-46c2-af6f-9bb1f489e3c4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:55.440168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1343/2668] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:15:55 2022] PUT /identity/v3/projects/6e5646f63528416dadefa845e1c80ee8/users/d8f01735f81f4ef3bc782a60f7c3bc3a/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:15:55.444502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521a56f-2932-43cb-acb7-fd9e931d332f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:55.455653 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521a56f-2932-43cb-acb7-fd9e931d332f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:55.517986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e1930-f2bc-402a-b97c-e3a0ac0de821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:55.518624 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e1930-f2bc-402a-b97c-e3a0ac0de821 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:55.518904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e1930-f2bc-402a-b97c-e3a0ac0de821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:55.519178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e1930-f2bc-402a-b97c-e3a0ac0de821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:55.519914 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e1930-f2bc-402a-b97c-e3a0ac0de821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:55.521537 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e1930-f2bc-402a-b97c-e3a0ac0de821 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:55.533250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1326/2669] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:55.537573 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e57c4e-19d7-46c2-af6f-9bb1f489e3c4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:55.549380 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e57c4e-19d7-46c2-af6f-9bb1f489e3c4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:55.555459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d5a1b-f87d-4417-aef0-afbb24f83d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:55.556361 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d5a1b-f87d-4417-aef0-afbb24f83d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:55.556639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d5a1b-f87d-4417-aef0-afbb24f83d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:55.557295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d5a1b-f87d-4417-aef0-afbb24f83d88 admin admin] PATH_INFO: `/v3/projects/6e5646f63528416dadefa845e1c80ee8/users/d8f01735f81f4ef3bc782a60f7c3bc3a/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:55.559890 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d5a1b-f87d-4417-aef0-afbb24f83d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5646f63528416dadefa845e1c80ee8, user_id=d8f01735f81f4ef3bc782a60f7c3bc3a, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:55.562600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d5a1b-f87d-4417-aef0-afbb24f83d88 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:55.573799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1344/2670] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:15:55 2022] PUT /identity/v3/projects/6e5646f63528416dadefa845e1c80ee8/users/d8f01735f81f4ef3bc782a60f7c3bc3a/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:55.584358 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252da5a-7a0d-4537-a21a-9e262c0e0f51 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:55.584626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252da5a-7a0d-4537-a21a-9e262c0e0f51 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:55.584938 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252da5a-7a0d-4537-a21a-9e262c0e0f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:55.623734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-8252da5a-7a0d-4537-a21a-9e262c0e0f51 None None] MFA Rules not processed for user `d8f01735f81f4ef3bc782a60f7c3bc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:55.624816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8252da5a-7a0d-4537-a21a-9e262c0e0f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:55.699410 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1327/2671] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:55.712748 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d5a1b-f87d-4417-aef0-afbb24f83d88 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:55.714609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-df7d5a1b-f87d-4417-aef0-afbb24f83d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:55.776797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d5a1b-f87d-4417-aef0-afbb24f83d88 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:55.780955 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3f25b-cd5c-4cd4-a23d-4d4c1222d1f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:55.781619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3f25b-cd5c-4cd4-a23d-4d4c1222d1f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:55.781892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3f25b-cd5c-4cd4-a23d-4d4c1222d1f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:55.782188 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3f25b-cd5c-4cd4-a23d-4d4c1222d1f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:55.793156 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3f25b-cd5c-4cd4-a23d-4d4c1222d1f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:55.794642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3f25b-cd5c-4cd4-a23d-4d4c1222d1f4 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:55.805362 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1345/2672] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:15:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:15:56.413606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252da5a-7a0d-4537-a21a-9e262c0e0f51 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:56.455929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:56.484920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252da5a-7a0d-4537-a21a-9e262c0e0f51 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:56.496263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:56.497163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:56.497441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:56.497714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] PATH_INFO: `/v3/users/a57c2c2857504b968014752f568e775f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:56.499063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a57c2c2857504b968014752f568e775f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:56.501042 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:56.561956 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a57c2c2857504b968014752f568e775f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:56.562665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a57c2c2857504b968014752f568e775f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:56.571875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a57c2c2857504b968014752f568e775f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:56.572514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a57c2c2857504b968014752f568e775f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:56.580690 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a57c2c2857504b968014752f568e775f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:56.601840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1328/2673] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:15:56 2022] DELETE /identity/v3/users/a57c2c2857504b968014752f568e775f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:56.607138 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62857e-0875-4ef1-abfe-1529fe731026 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:56.607682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62857e-0875-4ef1-abfe-1529fe731026 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:56.607682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62857e-0875-4ef1-abfe-1529fe731026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:56.626381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9b62857e-0875-4ef1-abfe-1529fe731026 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:56.627594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9b62857e-0875-4ef1-abfe-1529fe731026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:56.692066 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1346/2674] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:56.707427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:56.709073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:56.771626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548791f4-a5e0-42a6-bf99-9703023dd491 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:56.775864 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b114f-e358-45c5-a476-73e53e237150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:56.776608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b114f-e358-45c5-a476-73e53e237150 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:56.776933 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b114f-e358-45c5-a476-73e53e237150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:56.777214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b114f-e358-45c5-a476-73e53e237150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:56.787506 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b114f-e358-45c5-a476-73e53e237150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:56.788968 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b114f-e358-45c5-a476-73e53e237150 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:56.799904 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1329/2675] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:15:57.134038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62857e-0875-4ef1-abfe-1529fe731026 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:57.135723 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9b62857e-0875-4ef1-abfe-1529fe731026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:57.156155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62857e-0875-4ef1-abfe-1529fe731026 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:57.162979 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:57.163878 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:57.164154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:57.164428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] PATH_INFO: `/v3/projects/08c8b792095f4f9a80263ac9ee2dc33a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:57.165856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c8b792095f4f9a80263ac9ee2dc33a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:57.167962 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:57.176915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:57.221936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c8b792095f4f9a80263ac9ee2dc33a', 'request_id': 'req-2053fa31-4773-4916-82ee-03880f524683'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.230355 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c8b792095f4f9a80263ac9ee2dc33a', 'request_id': 'req-2053fa31-4773-4916-82ee-03880f524683'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.236845 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c8b792095f4f9a80263ac9ee2dc33a', 'request_id': 'req-2053fa31-4773-4916-82ee-03880f524683'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.249884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] The token cache is being invalidate because project 08c8b792095f4f9a80263ac9ee2dc33a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:15:57.250388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.251344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.253643 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1347/2676] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:15:57 2022] DELETE /identity/v3/projects/08c8b792095f4f9a80263ac9ee2dc33a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:57.258892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b114f-e358-45c5-a476-73e53e237150 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:57.269745 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b114f-e358-45c5-a476-73e53e237150 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:57.328797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:57.335830 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:57.336742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:57.337023 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:57.337304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] PATH_INFO: `/v3/users/d8f01735f81f4ef3bc782a60f7c3bc3a` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:57.338724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8f01735f81f4ef3bc782a60f7c3bc3a)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:57.342107 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:57.397296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8f01735f81f4ef3bc782a60f7c3bc3a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.397957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8f01735f81f4ef3bc782a60f7c3bc3a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.407974 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8f01735f81f4ef3bc782a60f7c3bc3a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.408788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8f01735f81f4ef3bc782a60f7c3bc3a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.419013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6bd27d0-3ba2-492a-92ec-632982c4a604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8f01735f81f4ef3bc782a60f7c3bc3a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.441461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1330/2677] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:15:57 2022] DELETE /identity/v3/users/d8f01735f81f4ef3bc782a60f7c3bc3a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:57.745151 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:57.746730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:57.764550 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053fa31-4773-4916-82ee-03880f524683 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:57.829039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:57.829922 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:57.830222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:57.830495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] PATH_INFO: `/v3/projects/6e5646f63528416dadefa845e1c80ee8` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:57.831727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5646f63528416dadefa845e1c80ee8)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:57.833642 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:57.886628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5646f63528416dadefa845e1c80ee8', 'request_id': 'req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.896866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5646f63528416dadefa845e1c80ee8', 'request_id': 'req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.902969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5646f63528416dadefa845e1c80ee8', 'request_id': 'req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.925438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] The token cache is being invalidate because project 6e5646f63528416dadefa845e1c80ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:15:57.925438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.926152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4b1bf711-c0fa-44f8-9dfb-e4270bebae36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:15:57.927925 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1348/2678] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:15:57 2022] DELETE /identity/v3/projects/6e5646f63528416dadefa845e1c80ee8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:57.934871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d0b88-baf1-4b0d-bee2-7b1b2e1e304e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:57.936462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d0b88-baf1-4b0d-bee2-7b1b2e1e304e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:57.937104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d0b88-baf1-4b0d-bee2-7b1b2e1e304e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:57.961636 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-515d0b88-baf1-4b0d-bee2-7b1b2e1e304e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:57.963126 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-515d0b88-baf1-4b0d-bee2-7b1b2e1e304e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:58.041332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1331/2679] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:58.051687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4030bf-a77d-4c45-af99-563d8491abb8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:58.052071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4030bf-a77d-4c45-af99-563d8491abb8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:58.052416 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4030bf-a77d-4c45-af99-563d8491abb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:58.073938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7a4030bf-a77d-4c45-af99-563d8491abb8 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:58.075196 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7a4030bf-a77d-4c45-af99-563d8491abb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:58.088539 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1349/2680] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:58.093589 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d0b88-baf1-4b0d-bee2-7b1b2e1e304e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:58.108230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d0b88-baf1-4b0d-bee2-7b1b2e1e304e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:58.113470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d14ba-cf55-4b09-b20c-259ef10b78c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:58.114172 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d14ba-cf55-4b09-b20c-259ef10b78c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:58.114466 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d14ba-cf55-4b09-b20c-259ef10b78c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:58.114721 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d14ba-cf55-4b09-b20c-259ef10b78c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:58.115458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d14ba-cf55-4b09-b20c-259ef10b78c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:15:58.115764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d14ba-cf55-4b09-b20c-259ef10b78c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:58.117362 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3d14ba-cf55-4b09-b20c-259ef10b78c9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:58.131866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1332/2681] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:15:58.136976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4030bf-a77d-4c45-af99-563d8491abb8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:58.151867 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4030bf-a77d-4c45-af99-563d8491abb8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:58.158588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814b823-7f9c-43c0-bca9-4b955bb630bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:58.159265 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814b823-7f9c-43c0-bca9-4b955bb630bd admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:58.159545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814b823-7f9c-43c0-bca9-4b955bb630bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:58.159818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814b823-7f9c-43c0-bca9-4b955bb630bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:58.160514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814b823-7f9c-43c0-bca9-4b955bb630bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:58.162478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814b823-7f9c-43c0-bca9-4b955bb630bd admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:58.178409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:58.198393 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1350/2682] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:15:58 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:15:58.204092 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d14ba-cf55-4b09-b20c-259ef10b78c9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:58.219250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d14ba-cf55-4b09-b20c-259ef10b78c9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:58.302350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d6da7-888a-494d-8a9f-a3a0d0f5782b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:58.303046 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6da7-888a-494d-8a9f-a3a0d0f5782b admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:58.303331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6da7-888a-494d-8a9f-a3a0d0f5782b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:58.303614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6da7-888a-494d-8a9f-a3a0d0f5782b admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:58.304315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d6da7-888a-494d-8a9f-a3a0d0f5782b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:58.306323 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d6da7-888a-494d-8a9f-a3a0d0f5782b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:58.316526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:58.333520 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1333/2683] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:15:58 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:15:58.337931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814b823-7f9c-43c0-bca9-4b955bb630bd admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:58.350435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814b823-7f9c-43c0-bca9-4b955bb630bd admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:58.356646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b7eb5-25e9-4d32-a510-75941ca72d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:58.357327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b7eb5-25e9-4d32-a510-75941ca72d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:58.357609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b7eb5-25e9-4d32-a510-75941ca72d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:58.358435 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b7eb5-25e9-4d32-a510-75941ca72d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:58.359126 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b7eb5-25e9-4d32-a510-75941ca72d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:58.360753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b7eb5-25e9-4d32-a510-75941ca72d4d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:58.372328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1351/2684] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:58.376579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d6da7-888a-494d-8a9f-a3a0d0f5782b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:58.387160 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d6da7-888a-494d-8a9f-a3a0d0f5782b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:58.392439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb55df-b65e-4b82-8654-19811f1934fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:58.393122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb55df-b65e-4b82-8654-19811f1934fe admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:58.393400 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb55df-b65e-4b82-8654-19811f1934fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:58.393671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb55df-b65e-4b82-8654-19811f1934fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:58.394444 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb55df-b65e-4b82-8654-19811f1934fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:58.396051 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb55df-b65e-4b82-8654-19811f1934fe admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:58.407657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1334/2685] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:58.412375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b7eb5-25e9-4d32-a510-75941ca72d4d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:58.426275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b7eb5-25e9-4d32-a510-75941ca72d4d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:58.431581 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67eeeb-e820-48c7-97f4-a72922857f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:58.432384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67eeeb-e820-48c7-97f4-a72922857f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:58.432659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67eeeb-e820-48c7-97f4-a72922857f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:58.432926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67eeeb-e820-48c7-97f4-a72922857f56 admin admin] PATH_INFO: `/v3/projects/0f1966a8a6994671968e629c892e77b4/users/43b0b1fcd55b417a956ec5f19c119025/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:58.453051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67eeeb-e820-48c7-97f4-a72922857f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f1966a8a6994671968e629c892e77b4, user_id=43b0b1fcd55b417a956ec5f19c119025, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:58.455769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67eeeb-e820-48c7-97f4-a72922857f56 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:58.476192 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1352/2686] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:15:58 2022] PUT /identity/v3/projects/0f1966a8a6994671968e629c892e77b4/users/43b0b1fcd55b417a956ec5f19c119025/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:58.482637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce9f20-35d9-41ac-be8d-740f4fdba19c None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:58.483912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce9f20-35d9-41ac-be8d-740f4fdba19c None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:58.483912 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce9f20-35d9-41ac-be8d-740f4fdba19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:58.506080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-67ce9f20-35d9-41ac-be8d-740f4fdba19c None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:58.507381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-67ce9f20-35d9-41ac-be8d-740f4fdba19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:58.579840 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1335/2687] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:58.590106 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67eeeb-e820-48c7-97f4-a72922857f56 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:58.592171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3e67eeeb-e820-48c7-97f4-a72922857f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:58.657517 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67eeeb-e820-48c7-97f4-a72922857f56 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:58.661629 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3e80-fc0f-45db-ad8c-ef94e9adc854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:58.662307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3e80-fc0f-45db-ad8c-ef94e9adc854 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:58.662655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3e80-fc0f-45db-ad8c-ef94e9adc854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:58.662885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3e80-fc0f-45db-ad8c-ef94e9adc854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:58.674059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc3e80-fc0f-45db-ad8c-ef94e9adc854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:58.675545 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc3e80-fc0f-45db-ad8c-ef94e9adc854 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:58.686209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1353/2688] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:15:59.331504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc3fc9-bcf7-4af6-b088-abf12fecc9c3 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:59.331861 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc3fc9-bcf7-4af6-b088-abf12fecc9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:59.332221 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc3fc9-bcf7-4af6-b088-abf12fecc9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:59.362687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-61bc3fc9-bcf7-4af6-b088-abf12fecc9c3 None None] MFA Rules not processed for user `43b0b1fcd55b417a956ec5f19c119025`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:15:59.363979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-61bc3fc9-bcf7-4af6-b088-abf12fecc9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:15:59.441064 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1336/2689] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:15:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:15:59.445839 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3e80-fc0f-45db-ad8c-ef94e9adc854 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:59.456089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3e80-fc0f-45db-ad8c-ef94e9adc854 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:59.460939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c05eb4-4594-4af8-92a4-84f20ff32c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:59.461684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05eb4-4594-4af8-92a4-84f20ff32c3c admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:59.462028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05eb4-4594-4af8-92a4-84f20ff32c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:59.462398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05eb4-4594-4af8-92a4-84f20ff32c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:59.463176 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c05eb4-4594-4af8-92a4-84f20ff32c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:59.465154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c05eb4-4594-4af8-92a4-84f20ff32c3c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:59.496866 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1354/2690] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:15:59 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:15:59.501011 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc3fc9-bcf7-4af6-b088-abf12fecc9c3 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:59.512735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc3fc9-bcf7-4af6-b088-abf12fecc9c3 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:59.582105 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bafd8-9a02-482d-b008-ea7b2f39b50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:59.583262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bafd8-9a02-482d-b008-ea7b2f39b50a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:59.583740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bafd8-9a02-482d-b008-ea7b2f39b50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:59.584215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bafd8-9a02-482d-b008-ea7b2f39b50a admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:59.585144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bafd8-9a02-482d-b008-ea7b2f39b50a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:59.587118 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bafd8-9a02-482d-b008-ea7b2f39b50a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:59.614427 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1337/2691] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:15:59 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:15:59.616312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c05eb4-4594-4af8-92a4-84f20ff32c3c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:59.629455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c05eb4-4594-4af8-92a4-84f20ff32c3c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:59.639827 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3f4f7-d118-48d1-b36c-adcf0e09c895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:59.640846 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3f4f7-d118-48d1-b36c-adcf0e09c895 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:59.641275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3f4f7-d118-48d1-b36c-adcf0e09c895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:59.641721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3f4f7-d118-48d1-b36c-adcf0e09c895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:59.642882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3f4f7-d118-48d1-b36c-adcf0e09c895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:59.644523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3f4f7-d118-48d1-b36c-adcf0e09c895 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:59.660269 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1355/2692] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:59.664642 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bafd8-9a02-482d-b008-ea7b2f39b50a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:59.670209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:15:59.677951 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bafd8-9a02-482d-b008-ea7b2f39b50a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:59.684035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990b3ae-50f9-4632-a4e5-e268633d38af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:59.684944 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b3ae-50f9-4632-a4e5-e268633d38af admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:59.685225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b3ae-50f9-4632-a4e5-e268633d38af admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:59.685501 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990b3ae-50f9-4632-a4e5-e268633d38af admin admin] PATH_INFO: `/v3/projects/36c43bbb2d1e4862b42158b726024aeb/users/d8c6104dc7cd4a56adcc081843e7df32/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:59.702737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990b3ae-50f9-4632-a4e5-e268633d38af admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c43bbb2d1e4862b42158b726024aeb, user_id=d8c6104dc7cd4a56adcc081843e7df32, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:59.705969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990b3ae-50f9-4632-a4e5-e268633d38af admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:59.720788 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1338/2693] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:15:59 2022] PUT /identity/v3/projects/36c43bbb2d1e4862b42158b726024aeb/users/d8c6104dc7cd4a56adcc081843e7df32/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:15:59.725725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3f4f7-d118-48d1-b36c-adcf0e09c895 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:59.737559 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3f4f7-d118-48d1-b36c-adcf0e09c895 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:59.807353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339e000-d9ad-4cb2-b2e1-5d66fd17ad2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:59.808034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e000-d9ad-4cb2-b2e1-5d66fd17ad2a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:59.808660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e000-d9ad-4cb2-b2e1-5d66fd17ad2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:59.808931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e000-d9ad-4cb2-b2e1-5d66fd17ad2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:59.809795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339e000-d9ad-4cb2-b2e1-5d66fd17ad2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:59.811828 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339e000-d9ad-4cb2-b2e1-5d66fd17ad2a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:59.822896 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1356/2694] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:15:59.827659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990b3ae-50f9-4632-a4e5-e268633d38af admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:15:59.839224 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990b3ae-50f9-4632-a4e5-e268633d38af admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:15:59.844197 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd57b55-37a9-4bd2-9d30-5ef174996343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:15:59.845091 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd57b55-37a9-4bd2-9d30-5ef174996343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:15:59.845371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd57b55-37a9-4bd2-9d30-5ef174996343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:15:59.845641 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd57b55-37a9-4bd2-9d30-5ef174996343 admin admin] PATH_INFO: `/v3/projects/36c43bbb2d1e4862b42158b726024aeb/users/d8c6104dc7cd4a56adcc081843e7df32/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:15:59.847731 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd57b55-37a9-4bd2-9d30-5ef174996343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c43bbb2d1e4862b42158b726024aeb, user_id=d8c6104dc7cd4a56adcc081843e7df32, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:15:59.850856 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd57b55-37a9-4bd2-9d30-5ef174996343 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:15:59.862517 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1339/2695] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:15:59 2022] PUT /identity/v3/projects/36c43bbb2d1e4862b42158b726024aeb/users/d8c6104dc7cd4a56adcc081843e7df32/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:16:00.499310 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46038bad-04d1-48ec-a60a-d69261c5c6fe None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:00.499910 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46038bad-04d1-48ec-a60a-d69261c5c6fe None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:00.500979 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46038bad-04d1-48ec-a60a-d69261c5c6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:00.528769 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-46038bad-04d1-48ec-a60a-d69261c5c6fe None None] MFA Rules not processed for user `d8c6104dc7cd4a56adcc081843e7df32`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:16:00.530003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-46038bad-04d1-48ec-a60a-d69261c5c6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:00.544392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:00.614852 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1357/2696] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:16:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:16:00.622835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd57b55-37a9-4bd2-9d30-5ef174996343 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:00.625436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-2fd57b55-37a9-4bd2-9d30-5ef174996343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:00.695649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd57b55-37a9-4bd2-9d30-5ef174996343 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:00.700385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d447d-5510-4f6d-b517-1c6458790e72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:00.701199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d447d-5510-4f6d-b517-1c6458790e72 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:00.701571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d447d-5510-4f6d-b517-1c6458790e72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:00.701950 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d447d-5510-4f6d-b517-1c6458790e72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:00.777033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d447d-5510-4f6d-b517-1c6458790e72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:00.778704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702d447d-5510-4f6d-b517-1c6458790e72 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:00.793917 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1340/2697] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:16:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:16:02.210307 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46038bad-04d1-48ec-a60a-d69261c5c6fe None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:02.211918 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-46038bad-04d1-48ec-a60a-d69261c5c6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:02.225043 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46038bad-04d1-48ec-a60a-d69261c5c6fe None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:02.228933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5f0eb-d023-4f73-a166-dd9fa2b51e3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:02.229597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5f0eb-d023-4f73-a166-dd9fa2b51e3e service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:02.229875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5f0eb-d023-4f73-a166-dd9fa2b51e3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:02.232341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5f0eb-d023-4f73-a166-dd9fa2b51e3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:02.240913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5f0eb-d023-4f73-a166-dd9fa2b51e3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:02.242408 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5f0eb-d023-4f73-a166-dd9fa2b51e3e service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:02.253685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1358/2698] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:16:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:16:10.785104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=62,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:12.248405 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=28 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:14.934381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d447d-5510-4f6d-b517-1c6458790e72 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:14.935969 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-702d447d-5510-4f6d-b517-1c6458790e72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:15.020024 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d447d-5510-4f6d-b517-1c6458790e72 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:15.025048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279eb1a-01e7-4044-99c8-898c28f5cb11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:15.025720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279eb1a-01e7-4044-99c8-898c28f5cb11 service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:15.026000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279eb1a-01e7-4044-99c8-898c28f5cb11 service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:15.026305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279eb1a-01e7-4044-99c8-898c28f5cb11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:15.028097 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0279eb1a-01e7-4044-99c8-898c28f5cb11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:15.040997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279eb1a-01e7-4044-99c8-898c28f5cb11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:15.042381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0279eb1a-01e7-4044-99c8-898c28f5cb11 service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:15.053403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1341/2699] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:16:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:16:15.563275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5f0eb-d023-4f73-a166-dd9fa2b51e3e service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:15.632366 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5f0eb-d023-4f73-a166-dd9fa2b51e3e service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:15.637134 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:15.638020 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:15.638326 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:15.638599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] PATH_INFO: `/v3/users/43b0b1fcd55b417a956ec5f19c119025` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:15.639802 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b0b1fcd55b417a956ec5f19c119025)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:15.641744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:15.689676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b0b1fcd55b417a956ec5f19c119025'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:15.697902 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b0b1fcd55b417a956ec5f19c119025'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:15.698540 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b0b1fcd55b417a956ec5f19c119025'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:15.699127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b0b1fcd55b417a956ec5f19c119025'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:15.711746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b0b1fcd55b417a956ec5f19c119025'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:15.724536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1359/2700] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:16:15 2022] DELETE /identity/v3/users/43b0b1fcd55b417a956ec5f19c119025 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:16:15.908181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279eb1a-01e7-4044-99c8-898c28f5cb11 service placement] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:15.924485 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279eb1a-01e7-4044-99c8-898c28f5cb11 service placement] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:15.994915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:15.995812 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:15.996096 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:15.996372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] PATH_INFO: `/v3/projects/0f1966a8a6994671968e629c892e77b4` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:15.997642 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f1966a8a6994671968e629c892e77b4)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:15.999634 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:16.049549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f1966a8a6994671968e629c892e77b4', 'request_id': 'req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.058449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f1966a8a6994671968e629c892e77b4', 'request_id': 'req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.066434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f1966a8a6994671968e629c892e77b4', 'request_id': 'req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.079113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] The token cache is being invalidate because project 0f1966a8a6994671968e629c892e77b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:16:16.079502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.080204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.081764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1342/2701] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:16:15 2022] DELETE /identity/v3/projects/0f1966a8a6994671968e629c892e77b4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:16:16.086329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:16.087780 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:16.102105 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c105e1-fed9-42f2-bca1-710a8ebf1190 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:16.163743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:16.164639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:16.164921 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:16.165195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] PATH_INFO: `/v3/users/d8c6104dc7cd4a56adcc081843e7df32` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:16.166438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c6104dc7cd4a56adcc081843e7df32)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:16.168381 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:16.194207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:16.224120 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c6104dc7cd4a56adcc081843e7df32'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.233133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c6104dc7cd4a56adcc081843e7df32'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.233746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c6104dc7cd4a56adcc081843e7df32'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.234429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c6104dc7cd4a56adcc081843e7df32'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.248158 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-45081304-50a0-4004-9f1b-421901d4a77b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c6104dc7cd4a56adcc081843e7df32'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.261100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1360/2702] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:16:16 2022] DELETE /identity/v3/users/d8c6104dc7cd4a56adcc081843e7df32 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:16:16.453626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:16.455132 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:16.471008 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4f3e0-41fd-4d3e-a808-17ed7c9ef5a1 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:16.490767 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:16.534596 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:16.536117 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:16.536396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:16.536678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] PATH_INFO: `/v3/projects/36c43bbb2d1e4862b42158b726024aeb` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:16.537942 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=36c43bbb2d1e4862b42158b726024aeb)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:16.539894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:16.592813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36c43bbb2d1e4862b42158b726024aeb', 'request_id': 'req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.602016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36c43bbb2d1e4862b42158b726024aeb', 'request_id': 'req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.608707 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36c43bbb2d1e4862b42158b726024aeb', 'request_id': 'req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.620820 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] The token cache is being invalidate because project 36c43bbb2d1e4862b42158b726024aeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:16:16.621111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.621896 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7592926f-6ef4-4c6e-b328-d8c287bcdb7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:16.623643 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1343/2703] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:16:16 2022] DELETE /identity/v3/projects/36c43bbb2d1e4862b42158b726024aeb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:16:16.629060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12a0e0-2e73-48d7-a5f4-09c36d7d1aa6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:16.629627 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12a0e0-2e73-48d7-a5f4-09c36d7d1aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:16.629627 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12a0e0-2e73-48d7-a5f4-09c36d7d1aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:16.649689 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bf12a0e0-2e73-48d7-a5f4-09c36d7d1aa6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:16:16.653611 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bf12a0e0-2e73-48d7-a5f4-09c36d7d1aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:16.735628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1361/2704] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:16:16.743875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948add2-fcb1-4cf4-baf4-e924a9c2e598 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:16.744123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948add2-fcb1-4cf4-baf4-e924a9c2e598 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:16.744431 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948add2-fcb1-4cf4-baf4-e924a9c2e598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:16.767035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7948add2-fcb1-4cf4-baf4-e924a9c2e598 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:16:16.768281 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7948add2-fcb1-4cf4-baf4-e924a9c2e598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:16.778585 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1344/2705] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:16:16.783104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12a0e0-2e73-48d7-a5f4-09c36d7d1aa6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:16.793754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12a0e0-2e73-48d7-a5f4-09c36d7d1aa6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:16.801166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df361b-1175-4572-b516-f37c70b62e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:16.801848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df361b-1175-4572-b516-f37c70b62e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:16.802133 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df361b-1175-4572-b516-f37c70b62e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:16.802433 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df361b-1175-4572-b516-f37c70b62e34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:16.803218 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df361b-1175-4572-b516-f37c70b62e34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:16:16.803528 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df361b-1175-4572-b516-f37c70b62e34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:16.805161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df361b-1175-4572-b516-f37c70b62e34 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:16.821440 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1362/2706] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 11:16:16.826218 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948add2-fcb1-4cf4-baf4-e924a9c2e598 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:16.836106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948add2-fcb1-4cf4-baf4-e924a9c2e598 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:16.842201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fb224-2e37-4ebb-b81e-25123565e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:16.842862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fb224-2e37-4ebb-b81e-25123565e6bd admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:16.843144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fb224-2e37-4ebb-b81e-25123565e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:16.843415 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fb224-2e37-4ebb-b81e-25123565e6bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:16.844093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fb224-2e37-4ebb-b81e-25123565e6bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:16.846016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fb224-2e37-4ebb-b81e-25123565e6bd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:16.875633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1345/2707] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:16:16 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:16:16.881797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df361b-1175-4572-b516-f37c70b62e34 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:16.892602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df361b-1175-4572-b516-f37c70b62e34 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:16.960090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca857fca-8e12-4fb2-9dbf-ec51f4fecc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:16.960779 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca857fca-8e12-4fb2-9dbf-ec51f4fecc42 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:16.961089 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca857fca-8e12-4fb2-9dbf-ec51f4fecc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:16.961365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca857fca-8e12-4fb2-9dbf-ec51f4fecc42 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:16.962064 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca857fca-8e12-4fb2-9dbf-ec51f4fecc42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:16.964071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca857fca-8e12-4fb2-9dbf-ec51f4fecc42 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:16.987533 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1363/2708] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:16:16 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:16:16.991775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fb224-2e37-4ebb-b81e-25123565e6bd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:17.004729 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fb224-2e37-4ebb-b81e-25123565e6bd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:17.010041 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f0e5f-fb53-47cb-8025-b25ea3f873be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:17.010757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f0e5f-fb53-47cb-8025-b25ea3f873be admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.011040 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f0e5f-fb53-47cb-8025-b25ea3f873be admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.011302 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f0e5f-fb53-47cb-8025-b25ea3f873be admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.012090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f0e5f-fb53-47cb-8025-b25ea3f873be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:17.013731 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f0e5f-fb53-47cb-8025-b25ea3f873be admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:17.027556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1346/2709] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:16:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:16:17.032537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca857fca-8e12-4fb2-9dbf-ec51f4fecc42 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:17.048238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca857fca-8e12-4fb2-9dbf-ec51f4fecc42 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:17.054708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606b6d0-fc4a-4bb0-be6c-7d7fa57fe94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:17.055392 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606b6d0-fc4a-4bb0-be6c-7d7fa57fe94e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.055672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606b6d0-fc4a-4bb0-be6c-7d7fa57fe94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.055953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606b6d0-fc4a-4bb0-be6c-7d7fa57fe94e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.057353 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606b6d0-fc4a-4bb0-be6c-7d7fa57fe94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:17.058824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606b6d0-fc4a-4bb0-be6c-7d7fa57fe94e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:17.070542 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1364/2710] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:16:17.075000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f0e5f-fb53-47cb-8025-b25ea3f873be admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:17.086226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f0e5f-fb53-47cb-8025-b25ea3f873be admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:17.091617 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5415-0232-4242-8ad0-05f434fc6756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:17.092905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5415-0232-4242-8ad0-05f434fc6756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.093242 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5415-0232-4242-8ad0-05f434fc6756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.093587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5415-0232-4242-8ad0-05f434fc6756 admin admin] PATH_INFO: `/v3/projects/e8e49154904d400b93c84829e3520edf/users/6b11afd28c7a48d184b6fce791690e9b/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.112496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd5415-0232-4242-8ad0-05f434fc6756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e49154904d400b93c84829e3520edf, user_id=6b11afd28c7a48d184b6fce791690e9b, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:17.115916 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd5415-0232-4242-8ad0-05f434fc6756 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:17.131992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1347/2711] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:16:17 2022] PUT /identity/v3/projects/e8e49154904d400b93c84829e3520edf/users/6b11afd28c7a48d184b6fce791690e9b/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:16:17.143445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c1fae-0900-4abd-a6b7-ac05b49a755a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.143819 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c1fae-0900-4abd-a6b7-ac05b49a755a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.144195 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c1fae-0900-4abd-a6b7-ac05b49a755a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.151865 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:17.174870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1d2c1fae-0900-4abd-a6b7-ac05b49a755a None None] MFA Rules not processed for user `6b11afd28c7a48d184b6fce791690e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:16:17.176626 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1d2c1fae-0900-4abd-a6b7-ac05b49a755a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:17.249467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1365/2712] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:16:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:16:17.255278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5415-0232-4242-8ad0-05f434fc6756 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:17.311911 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:17.332557 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5415-0232-4242-8ad0-05f434fc6756 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:17.339972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aefc0-f284-425e-aa2e-aa9f0194e021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:17.340648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aefc0-f284-425e-aa2e-aa9f0194e021 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.340918 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aefc0-f284-425e-aa2e-aa9f0194e021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.341188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aefc0-f284-425e-aa2e-aa9f0194e021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.341888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482aefc0-f284-425e-aa2e-aa9f0194e021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:17.343886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482aefc0-f284-425e-aa2e-aa9f0194e021 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:17.375332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1348/2713] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:16:17 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:16:17.380663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c1fae-0900-4abd-a6b7-ac05b49a755a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:17.393088 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c1fae-0900-4abd-a6b7-ac05b49a755a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:17.453125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d75c-b6a5-442a-a372-5fdd7eb76e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:17.453800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d75c-b6a5-442a-a372-5fdd7eb76e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.454080 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d75c-b6a5-442a-a372-5fdd7eb76e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.454388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d75c-b6a5-442a-a372-5fdd7eb76e7f admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.455092 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27d75c-b6a5-442a-a372-5fdd7eb76e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:17.457044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27d75c-b6a5-442a-a372-5fdd7eb76e7f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:17.479744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1366/2714] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:16:17 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:16:17.483991 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aefc0-f284-425e-aa2e-aa9f0194e021 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:17.494924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aefc0-f284-425e-aa2e-aa9f0194e021 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:17.499860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61759bf-62a4-46dc-8313-401fe31f090a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:17.500468 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61759bf-62a4-46dc-8313-401fe31f090a admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.500752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61759bf-62a4-46dc-8313-401fe31f090a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.501033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61759bf-62a4-46dc-8313-401fe31f090a admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.501781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61759bf-62a4-46dc-8313-401fe31f090a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:17.503456 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61759bf-62a4-46dc-8313-401fe31f090a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:17.515083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1349/2715] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:16:17.519623 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d75c-b6a5-442a-a372-5fdd7eb76e7f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:17.530539 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d75c-b6a5-442a-a372-5fdd7eb76e7f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:17.536014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b7175-ea0f-4d89-b803-768dc8177861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:17.536920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b7175-ea0f-4d89-b803-768dc8177861 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.537201 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b7175-ea0f-4d89-b803-768dc8177861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.537478 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b7175-ea0f-4d89-b803-768dc8177861 admin admin] PATH_INFO: `/v3/projects/97a75ea739cd4a05af35bfe9775b9a7f/users/e47a39f4c59844c5bed5f69a6b8bc916/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.554646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b7175-ea0f-4d89-b803-768dc8177861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a75ea739cd4a05af35bfe9775b9a7f, user_id=e47a39f4c59844c5bed5f69a6b8bc916, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:17.557343 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b7175-ea0f-4d89-b803-768dc8177861 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:17.572786 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1367/2716] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:16:17 2022] PUT /identity/v3/projects/97a75ea739cd4a05af35bfe9775b9a7f/users/e47a39f4c59844c5bed5f69a6b8bc916/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:16:17.577292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61759bf-62a4-46dc-8313-401fe31f090a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:17.587905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61759bf-62a4-46dc-8313-401fe31f090a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:17.654122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab786e-50ba-4090-b01b-0faad235ea0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:17.654826 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab786e-50ba-4090-b01b-0faad235ea0d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.655112 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab786e-50ba-4090-b01b-0faad235ea0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.655422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab786e-50ba-4090-b01b-0faad235ea0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.656165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab786e-50ba-4090-b01b-0faad235ea0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:17.657809 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab786e-50ba-4090-b01b-0faad235ea0d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:17.668758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1350/2717] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:16:17.673097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b7175-ea0f-4d89-b803-768dc8177861 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:17.685213 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b7175-ea0f-4d89-b803-768dc8177861 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:17.690130 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147d7bd-a2a9-489d-9868-cad7ccc16f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:17.691047 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147d7bd-a2a9-489d-9868-cad7ccc16f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.691541 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147d7bd-a2a9-489d-9868-cad7ccc16f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.691806 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147d7bd-a2a9-489d-9868-cad7ccc16f45 admin admin] PATH_INFO: `/v3/projects/97a75ea739cd4a05af35bfe9775b9a7f/users/e47a39f4c59844c5bed5f69a6b8bc916/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.693776 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147d7bd-a2a9-489d-9868-cad7ccc16f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a75ea739cd4a05af35bfe9775b9a7f, user_id=e47a39f4c59844c5bed5f69a6b8bc916, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:17.696500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147d7bd-a2a9-489d-9868-cad7ccc16f45 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:17.708344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1368/2718] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:16:17 2022] PUT /identity/v3/projects/97a75ea739cd4a05af35bfe9775b9a7f/users/e47a39f4c59844c5bed5f69a6b8bc916/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:16:17.719338 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4fcc0-76ca-47d7-8ee9-a8a8bbe837fe None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.719633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4fcc0-76ca-47d7-8ee9-a8a8bbe837fe None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.719943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4fcc0-76ca-47d7-8ee9-a8a8bbe837fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.747786 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-4fa4fcc0-76ca-47d7-8ee9-a8a8bbe837fe None None] MFA Rules not processed for user `e47a39f4c59844c5bed5f69a6b8bc916`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:16:17.749003 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4fa4fcc0-76ca-47d7-8ee9-a8a8bbe837fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:17.823495 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1351/2719] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:16:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:16:17.834935 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147d7bd-a2a9-489d-9868-cad7ccc16f45 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:17.836805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1147d7bd-a2a9-489d-9868-cad7ccc16f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:17.891707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:17.899811 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147d7bd-a2a9-489d-9868-cad7ccc16f45 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:17.903908 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031e1a7-666b-4c80-804d-4eb5a8dec8e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:17.904568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031e1a7-666b-4c80-804d-4eb5a8dec8e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:17.904847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031e1a7-666b-4c80-804d-4eb5a8dec8e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:17.905118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031e1a7-666b-4c80-804d-4eb5a8dec8e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:17.973285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031e1a7-666b-4c80-804d-4eb5a8dec8e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:17.974770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031e1a7-666b-4c80-804d-4eb5a8dec8e7 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:17.984496 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1369/2720] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:16:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:16:22.246319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4fcc0-76ca-47d7-8ee9-a8a8bbe837fe None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:22.248162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-4fa4fcc0-76ca-47d7-8ee9-a8a8bbe837fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:22.262801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4fcc0-76ca-47d7-8ee9-a8a8bbe837fe None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:22.271205 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589ce41-1e1a-4fd5-984d-ae3943f9a381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:22.272093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589ce41-1e1a-4fd5-984d-ae3943f9a381 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:22.272378 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589ce41-1e1a-4fd5-984d-ae3943f9a381 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:22.272655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589ce41-1e1a-4fd5-984d-ae3943f9a381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:22.286066 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589ce41-1e1a-4fd5-984d-ae3943f9a381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:22.287948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589ce41-1e1a-4fd5-984d-ae3943f9a381 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:22.299656 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1352/2721] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:16:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:16:27.257744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031e1a7-666b-4c80-804d-4eb5a8dec8e7 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:27.325658 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031e1a7-666b-4c80-804d-4eb5a8dec8e7 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:27.330590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:27.331489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:27.331754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:27.332029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] PATH_INFO: `/v3/users/6b11afd28c7a48d184b6fce791690e9b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:27.333209 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b11afd28c7a48d184b6fce791690e9b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:27.335265 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:27.385104 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b11afd28c7a48d184b6fce791690e9b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:27.393958 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b11afd28c7a48d184b6fce791690e9b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:27.394605 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b11afd28c7a48d184b6fce791690e9b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:27.395204 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b11afd28c7a48d184b6fce791690e9b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:27.407724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b11afd28c7a48d184b6fce791690e9b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:27.420100 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1370/2722] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:16:27 2022] DELETE /identity/v3/users/6b11afd28c7a48d184b6fce791690e9b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:16:27.424751 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7363a-8628-4d07-b927-23369edd46d2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:27.425230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7363a-8628-4d07-b927-23369edd46d2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:27.425327 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7363a-8628-4d07-b927-23369edd46d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:27.447104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-91c7363a-8628-4d07-b927-23369edd46d2 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:16:27.448335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-91c7363a-8628-4d07-b927-23369edd46d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:27.474825 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:27.514575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1353/2723] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:16:27.526797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:27.528390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:27.551949 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:27.594394 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb10da7-7c9d-42d5-9487-9bea76cccbff admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:27.598986 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aadb38-74da-423c-9e9a-16c0f5daa032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:27.599653 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aadb38-74da-423c-9e9a-16c0f5daa032 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:27.599927 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aadb38-74da-423c-9e9a-16c0f5daa032 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:27.600207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aadb38-74da-423c-9e9a-16c0f5daa032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:27.610685 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aadb38-74da-423c-9e9a-16c0f5daa032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:27.612150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aadb38-74da-423c-9e9a-16c0f5daa032 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:27.622704 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1371/2724] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:16:27.898584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7363a-8628-4d07-b927-23369edd46d2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:27.900063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-91c7363a-8628-4d07-b927-23369edd46d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:27.913871 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7363a-8628-4d07-b927-23369edd46d2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:27.920941 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:27.921858 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:27.922165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:27.922440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] PATH_INFO: `/v3/projects/e8e49154904d400b93c84829e3520edf` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:27.923733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e49154904d400b93c84829e3520edf)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:27.925733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:27.975104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e49154904d400b93c84829e3520edf', 'request_id': 'req-a7d32588-53f8-46e5-833e-a1d36524b8d2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:27.984420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e49154904d400b93c84829e3520edf', 'request_id': 'req-a7d32588-53f8-46e5-833e-a1d36524b8d2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:27.990492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e49154904d400b93c84829e3520edf', 'request_id': 'req-a7d32588-53f8-46e5-833e-a1d36524b8d2'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.003898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] The token cache is being invalidate because project e8e49154904d400b93c84829e3520edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:16:28.004188 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.004901 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.006711 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1354/2725] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:16:27 2022] DELETE /identity/v3/projects/e8e49154904d400b93c84829e3520edf => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:16:28.011001 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aadb38-74da-423c-9e9a-16c0f5daa032 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:28.021385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aadb38-74da-423c-9e9a-16c0f5daa032 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:28.081873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:28.082805 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:28.083091 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:28.083344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] PATH_INFO: `/v3/users/e47a39f4c59844c5bed5f69a6b8bc916` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:28.085181 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47a39f4c59844c5bed5f69a6b8bc916)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:28.086948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:28.147790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47a39f4c59844c5bed5f69a6b8bc916'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.155954 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47a39f4c59844c5bed5f69a6b8bc916'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.156571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47a39f4c59844c5bed5f69a6b8bc916'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.157170 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47a39f4c59844c5bed5f69a6b8bc916'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.174220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47a39f4c59844c5bed5f69a6b8bc916'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.191676 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1372/2726] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:16:28 2022] DELETE /identity/v3/users/e47a39f4c59844c5bed5f69a6b8bc916 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:16:28.495512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:28.496979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:28.513490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d32588-53f8-46e5-833e-a1d36524b8d2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:28.584559 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:28.586376 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:28.586659 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:28.586934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] PATH_INFO: `/v3/projects/97a75ea739cd4a05af35bfe9775b9a7f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:28.588305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a75ea739cd4a05af35bfe9775b9a7f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:28.590306 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:28.648750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a75ea739cd4a05af35bfe9775b9a7f', 'request_id': 'req-c6c76b81-77db-4305-bbab-9f02850f3c53'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.652367 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:28.657238 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a75ea739cd4a05af35bfe9775b9a7f', 'request_id': 'req-c6c76b81-77db-4305-bbab-9f02850f3c53'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.664149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a75ea739cd4a05af35bfe9775b9a7f', 'request_id': 'req-c6c76b81-77db-4305-bbab-9f02850f3c53'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.676417 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] The token cache is being invalidate because project 97a75ea739cd4a05af35bfe9775b9a7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:16:28.676678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.677381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:28.679039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1355/2727] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:16:28 2022] DELETE /identity/v3/projects/97a75ea739cd4a05af35bfe9775b9a7f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:16:38.187635 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=70,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:38.672985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=6,keystone:DELETE=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:40.099312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:40.101032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:40.163317 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adce7d6-58b3-4367-a65d-82f14ff4e867 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:40.216259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f0a62-fc00-4a2f-aa86-8c1562cb29b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:40.216936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f0a62-fc00-4a2f-aa86-8c1562cb29b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:40.217219 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f0a62-fc00-4a2f-aa86-8c1562cb29b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:40.217501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f0a62-fc00-4a2f-aa86-8c1562cb29b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:40.219425 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8c7f0a62-fc00-4a2f-aa86-8c1562cb29b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:40.303056 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f0a62-fc00-4a2f-aa86-8c1562cb29b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:40.304563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f0a62-fc00-4a2f-aa86-8c1562cb29b8 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:40.325139 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1373/2728] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:16:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:16:41.190147 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:41.192208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:41.280692 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c76b81-77db-4305-bbab-9f02850f3c53 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:41.286635 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5346312-1606-4f13-8109-24f0fc348734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:41.287352 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5346312-1606-4f13-8109-24f0fc348734 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:41.287613 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5346312-1606-4f13-8109-24f0fc348734 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:41.287891 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5346312-1606-4f13-8109-24f0fc348734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:41.290479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f5346312-1606-4f13-8109-24f0fc348734 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:41.304425 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5346312-1606-4f13-8109-24f0fc348734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:41.307059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5346312-1606-4f13-8109-24f0fc348734 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:41.326128 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1356/2729] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:16:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:16:43.681057 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f0a62-fc00-4a2f-aa86-8c1562cb29b8 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:43.683787 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8c7f0a62-fc00-4a2f-aa86-8c1562cb29b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:43.702053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f0a62-fc00-4a2f-aa86-8c1562cb29b8 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:43.708449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720f5f-78ca-48ea-a6da-0f6d0c766c10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:43.709121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720f5f-78ca-48ea-a6da-0f6d0c766c10 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:43.709420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720f5f-78ca-48ea-a6da-0f6d0c766c10 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:43.709934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720f5f-78ca-48ea-a6da-0f6d0c766c10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:43.713375 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0c720f5f-78ca-48ea-a6da-0f6d0c766c10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:43.732012 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c720f5f-78ca-48ea-a6da-0f6d0c766c10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:43.733741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c720f5f-78ca-48ea-a6da-0f6d0c766c10 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:43.746694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1374/2730] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:16:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:16:51.314563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=34 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:52.743320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5346312-1606-4f13-8109-24f0fc348734 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:52.745181 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f5346312-1606-4f13-8109-24f0fc348734 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:52.756649 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5346312-1606-4f13-8109-24f0fc348734 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:52.761663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969fa0d-2f2e-4838-a47a-82d81f2e6c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:52.762374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969fa0d-2f2e-4838-a47a-82d81f2e6c9c service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:52.762688 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969fa0d-2f2e-4838-a47a-82d81f2e6c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:52.762963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f969fa0d-2f2e-4838-a47a-82d81f2e6c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:52.765000 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f969fa0d-2f2e-4838-a47a-82d81f2e6c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:52.835678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969fa0d-2f2e-4838-a47a-82d81f2e6c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:52.837482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f969fa0d-2f2e-4838-a47a-82d81f2e6c9c service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:52.848807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1357/2731] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:16:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:16:53.740820 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=87 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:16:59.559230 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720f5f-78ca-48ea-a6da-0f6d0c766c10 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:16:59.560926 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0c720f5f-78ca-48ea-a6da-0f6d0c766c10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:16:59.658278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720f5f-78ca-48ea-a6da-0f6d0c766c10 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:16:59.664724 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:16:59.665904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:16:59.666189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:16:59.666473 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] PATH_INFO: `/v3/users/ee21ef9eacd24177a852f28d17376191` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:16:59.668189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee21ef9eacd24177a852f28d17376191)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:16:59.669894 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:16:59.747121 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee21ef9eacd24177a852f28d17376191'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:59.756333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee21ef9eacd24177a852f28d17376191'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:59.757065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee21ef9eacd24177a852f28d17376191'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:59.757741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee21ef9eacd24177a852f28d17376191'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:59.771409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee21ef9eacd24177a852f28d17376191'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:16:59.784404 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1375/2732] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:16:59 2022] DELETE /identity/v3/users/ee21ef9eacd24177a852f28d17376191 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:16:59.966195 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969fa0d-2f2e-4838-a47a-82d81f2e6c9c service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:17:00.026924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f969fa0d-2f2e-4838-a47a-82d81f2e6c9c service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:17:00.084758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:17:00.085739 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:17:00.086146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:17:00.086604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] PATH_INFO: `/v3/projects/17bb5c3721fc4505a51879dae4762b43` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:17:00.087822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17bb5c3721fc4505a51879dae4762b43)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:17:00.089743 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:17:00.140480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17bb5c3721fc4505a51879dae4762b43', 'request_id': 'req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.148780 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17bb5c3721fc4505a51879dae4762b43', 'request_id': 'req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.150167 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:17:00.156162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17bb5c3721fc4505a51879dae4762b43', 'request_id': 'req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.168558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] The token cache is being invalidate because project 17bb5c3721fc4505a51879dae4762b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:17:00.168849 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.169584 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.171233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1358/2733] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:16:59 2022] DELETE /identity/v3/projects/17bb5c3721fc4505a51879dae4762b43 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:17:00.175318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:17:00.176895 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:17:00.192125 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec145d2c-6734-416c-b567-73f69803f9d0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:17:00.252422 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:17:00.253777 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:17:00.254140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:17:00.254505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] PATH_INFO: `/v3/users/3217e3b38bb648cb98dafba657b8f72f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:17:00.275009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3217e3b38bb648cb98dafba657b8f72f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:17:00.277844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:17:00.279885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:17:00.335292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3217e3b38bb648cb98dafba657b8f72f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.346915 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3217e3b38bb648cb98dafba657b8f72f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.347590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3217e3b38bb648cb98dafba657b8f72f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.348292 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3217e3b38bb648cb98dafba657b8f72f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.364396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-db2eb04b-c377-4710-aaf0-4fa1bbaf22d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3217e3b38bb648cb98dafba657b8f72f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.379495 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1376/2734] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:17:00 2022] DELETE /identity/v3/users/3217e3b38bb648cb98dafba657b8f72f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:17:00.567987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:17:00.569823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:17:00.584629 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d345aa-dd4f-40a4-9ed3-832e96cca9d9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:17:00.656326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:17:00.657211 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:17:00.657493 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:17:00.657777 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] PATH_INFO: `/v3/projects/8d5fcf3b767a463083bb51756473c816` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:17:00.669957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d5fcf3b767a463083bb51756473c816)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:17:00.671943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:17:00.725351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d5fcf3b767a463083bb51756473c816', 'request_id': 'req-1688d425-5220-4dac-9b71-d4cfe4dec17f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.734310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d5fcf3b767a463083bb51756473c816', 'request_id': 'req-1688d425-5220-4dac-9b71-d4cfe4dec17f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.740389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d5fcf3b767a463083bb51756473c816', 'request_id': 'req-1688d425-5220-4dac-9b71-d4cfe4dec17f'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.760893 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] The token cache is being invalidate because project 8d5fcf3b767a463083bb51756473c816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:17:00.761180 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.761913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:17:00.763609 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1359/2735] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:17:00 2022] DELETE /identity/v3/projects/8d5fcf3b767a463083bb51756473c816 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:17:06.180801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24cccb-50e3-49d4-aef1-36984f3a9da4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:17:06.181334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24cccb-50e3-49d4-aef1-36984f3a9da4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:17:06.181334 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24cccb-50e3-49d4-aef1-36984f3a9da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:17:06.199662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2f24cccb-50e3-49d4-aef1-36984f3a9da4 None None] MFA Rules not processed for user `b7275026b00b4e93bed7682b3ff2525c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:17:06.200944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2f24cccb-50e3-49d4-aef1-36984f3a9da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:17:06.275409 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1377/2736] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:17:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:17:06.591119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:17:06.592661 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:17:06.668509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:17:06.695666 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688d425-5220-4dac-9b71-d4cfe4dec17f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:17:06.699545 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c808f-d77d-43d8-a123-7d268a8875b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:17:06.700199 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c808f-d77d-43d8-a123-7d268a8875b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:17:06.700482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c808f-d77d-43d8-a123-7d268a8875b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:17:06.700764 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c808f-d77d-43d8-a123-7d268a8875b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:17:06.702657 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-406c808f-d77d-43d8-a123-7d268a8875b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:17:06.712503 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c808f-d77d-43d8-a123-7d268a8875b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:17:06.713962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c808f-d77d-43d8-a123-7d268a8875b2 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:17:06.722009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1360/2737] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:17:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:17:07.059403 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24cccb-50e3-49d4-aef1-36984f3a9da4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:17:07.060901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2f24cccb-50e3-49d4-aef1-36984f3a9da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:17:07.134929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24cccb-50e3-49d4-aef1-36984f3a9da4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:17:07.139934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958748df-6278-44dc-b725-cea296d6d8b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:17:07.140585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958748df-6278-44dc-b725-cea296d6d8b6 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:17:07.140869 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958748df-6278-44dc-b725-cea296d6d8b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:17:07.141149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958748df-6278-44dc-b725-cea296d6d8b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:17:07.143004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-958748df-6278-44dc-b725-cea296d6d8b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:17:07.154352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958748df-6278-44dc-b725-cea296d6d8b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:17:07.155930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958748df-6278-44dc-b725-cea296d6d8b6 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:17:07.166204 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1378/2738] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:17:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:17:16.718515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=12 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:17:17.161718 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:17:33.345671 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c808f-d77d-43d8-a123-7d268a8875b2 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:17:33.371093 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c808f-d77d-43d8-a123-7d268a8875b2 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:17:33.376356 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2b159-bfea-456e-843a-6fe8834a13c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:17:33.377476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b159-bfea-456e-843a-6fe8834a13c8 service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:17:33.377753 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b159-bfea-456e-843a-6fe8834a13c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:17:33.378026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2b159-bfea-456e-843a-6fe8834a13c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:17:33.379981 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-37f2b159-bfea-456e-843a-6fe8834a13c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:17:33.407343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2b159-bfea-456e-843a-6fe8834a13c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:17:33.409563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2b159-bfea-456e-843a-6fe8834a13c8 service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:17:33.420462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1361/2739] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:17:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:17:43.415693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=15 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:18:25.087590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958748df-6278-44dc-b725-cea296d6d8b6 service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:18:25.089038 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-958748df-6278-44dc-b725-cea296d6d8b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:18:25.219327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958748df-6278-44dc-b725-cea296d6d8b6 service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:18:25.227110 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f169180-8fb7-4e09-9b42-00a6923ae622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:18:25.227893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f169180-8fb7-4e09-9b42-00a6923ae622 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:18:25.228149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f169180-8fb7-4e09-9b42-00a6923ae622 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:18:25.228428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f169180-8fb7-4e09-9b42-00a6923ae622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:18:25.231071 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6f169180-8fb7-4e09-9b42-00a6923ae622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:18:25.243032 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f169180-8fb7-4e09-9b42-00a6923ae622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:18:25.244709 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f169180-8fb7-4e09-9b42-00a6923ae622 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:18:25.254559 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1379/2740] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:18:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:18:35.249882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=46 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:18:43.312561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ac542-f4a7-4eee-8706-6db9c1cc9fec None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:18:43.313146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ac542-f4a7-4eee-8706-6db9c1cc9fec None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:18:43.313146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ac542-f4a7-4eee-8706-6db9c1cc9fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:18:43.333491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-d73ac542-f4a7-4eee-8706-6db9c1cc9fec None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:18:43.334757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d73ac542-f4a7-4eee-8706-6db9c1cc9fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:18:43.398102 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1362/2741] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:18:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:18:43.448588 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f169180-8fb7-4e09-9b42-00a6923ae622 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:18:43.450319 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6f169180-8fb7-4e09-9b42-00a6923ae622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:18:43.528518 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f169180-8fb7-4e09-9b42-00a6923ae622 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:18:43.534286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172344d-8441-47d6-830e-53b455e27638 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:18:43.534952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172344d-8441-47d6-830e-53b455e27638 service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:18:43.535234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172344d-8441-47d6-830e-53b455e27638 service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:18:43.535510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172344d-8441-47d6-830e-53b455e27638 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:18:43.544968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172344d-8441-47d6-830e-53b455e27638 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:18:43.546617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172344d-8441-47d6-830e-53b455e27638 service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:18:43.555324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1380/2742] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:18:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:18:53.390789 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=25 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:18:53.551557 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=32 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:18:56.260183 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ac542-f4a7-4eee-8706-6db9c1cc9fec None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:18:56.261798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d73ac542-f4a7-4eee-8706-6db9c1cc9fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:18:56.276090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ac542-f4a7-4eee-8706-6db9c1cc9fec None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:18:56.281405 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154ebd0-d233-4a5a-abb6-109e783d74dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:18:56.282429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154ebd0-d233-4a5a-abb6-109e783d74dc service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:18:56.282892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154ebd0-d233-4a5a-abb6-109e783d74dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:18:56.283288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154ebd0-d233-4a5a-abb6-109e783d74dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:18:56.286020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3154ebd0-d233-4a5a-abb6-109e783d74dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:18:56.299580 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154ebd0-d233-4a5a-abb6-109e783d74dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:18:56.301054 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154ebd0-d233-4a5a-abb6-109e783d74dc service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:18:56.310353 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1363/2743] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:18:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:18:56.404753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172344d-8441-47d6-830e-53b455e27638 service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:18:56.413534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172344d-8441-47d6-830e-53b455e27638 service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:18:56.417296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c4bc2-e79a-4076-af59-eeaf451a7c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:18:56.418079 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c4bc2-e79a-4076-af59-eeaf451a7c3d service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:18:56.418368 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c4bc2-e79a-4076-af59-eeaf451a7c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:18:56.418644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c4bc2-e79a-4076-af59-eeaf451a7c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:18:56.425522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b62c4bc2-e79a-4076-af59-eeaf451a7c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:18:56.440592 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c4bc2-e79a-4076-af59-eeaf451a7c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:18:56.442059 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c4bc2-e79a-4076-af59-eeaf451a7c3d service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:18:56.454746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1381/2744] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:18:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 11:18:58.107201 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1a6fe-57b8-487e-b99d-4eeab038db3d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:18:58.107728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1a6fe-57b8-487e-b99d-4eeab038db3d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:18:58.107728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1a6fe-57b8-487e-b99d-4eeab038db3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:18:58.127377 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-23e1a6fe-57b8-487e-b99d-4eeab038db3d None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:18:58.128593 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-23e1a6fe-57b8-487e-b99d-4eeab038db3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:18:58.137315 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1364/2745] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:18:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:18:58.149510 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c4bc2-e79a-4076-af59-eeaf451a7c3d service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:18:58.161502 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c4bc2-e79a-4076-af59-eeaf451a7c3d service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:18:58.166942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7aafe-d754-402f-b4ea-5523aa9efc4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:18:58.167586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7aafe-d754-402f-b4ea-5523aa9efc4b service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:18:58.167929 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7aafe-d754-402f-b4ea-5523aa9efc4b service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:18:58.168367 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7aafe-d754-402f-b4ea-5523aa9efc4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:18:58.177238 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7aafe-d754-402f-b4ea-5523aa9efc4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:18:58.181600 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7aafe-d754-402f-b4ea-5523aa9efc4b service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:18:58.187747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1382/2746] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:18:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:19:08.121386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=9 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:19:08.184271 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=12 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:19:14.462760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288788b-ca83-4a0d-8794-233537d16dfb None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:19:14.463273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288788b-ca83-4a0d-8794-233537d16dfb None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:19:14.463395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288788b-ca83-4a0d-8794-233537d16dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:19:14.485434 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3288788b-ca83-4a0d-8794-233537d16dfb None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:19:14.486692 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3288788b-ca83-4a0d-8794-233537d16dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:19:14.494639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1365/2747] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jun 20 11:19:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:19:14.507604 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7aafe-d754-402f-b4ea-5523aa9efc4b service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:19:14.518976 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7aafe-d754-402f-b4ea-5523aa9efc4b service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:19:14.527058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a3bfd-a216-4af8-9ef3-d7d900aa5f23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:19:14.527720 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a3bfd-a216-4af8-9ef3-d7d900aa5f23 service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:19:14.528004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a3bfd-a216-4af8-9ef3-d7d900aa5f23 service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:19:14.528282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a3bfd-a216-4af8-9ef3-d7d900aa5f23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:19:14.537488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a3bfd-a216-4af8-9ef3-d7d900aa5f23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:19:14.539101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a3bfd-a216-4af8-9ef3-d7d900aa5f23 service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:19:14.548476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1383/2748] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:19:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:19:24.477621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:19:24.544328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=6 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:20:03.873300 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288788b-ca83-4a0d-8794-233537d16dfb None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:03.884490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288788b-ca83-4a0d-8794-233537d16dfb None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:03.888204 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39017ed-99c2-4e21-92b0-bc4dfe7aa9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:03.888872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39017ed-99c2-4e21-92b0-bc4dfe7aa9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:03.889159 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39017ed-99c2-4e21-92b0-bc4dfe7aa9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:03.889435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39017ed-99c2-4e21-92b0-bc4dfe7aa9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:03.891363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-d39017ed-99c2-4e21-92b0-bc4dfe7aa9d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:03.905648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39017ed-99c2-4e21-92b0-bc4dfe7aa9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:03.907152 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39017ed-99c2-4e21-92b0-bc4dfe7aa9d3 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:03.921626 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1366/2749] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:20:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:20:05.121880 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a3bfd-a216-4af8-9ef3-d7d900aa5f23 service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:05.133296 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a3bfd-a216-4af8-9ef3-d7d900aa5f23 service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:05.137398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724a495-e8c2-48f7-8c8b-86aa6ef1efbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:05.138033 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724a495-e8c2-48f7-8c8b-86aa6ef1efbf service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:05.138378 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724a495-e8c2-48f7-8c8b-86aa6ef1efbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:05.138657 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724a495-e8c2-48f7-8c8b-86aa6ef1efbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:05.147217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8724a495-e8c2-48f7-8c8b-86aa6ef1efbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:05.164567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724a495-e8c2-48f7-8c8b-86aa6ef1efbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:05.165946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724a495-e8c2-48f7-8c8b-86aa6ef1efbf service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:05.179419 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1384/2750] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:20:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:20:10.931816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39017ed-99c2-4e21-92b0-bc4dfe7aa9d3 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:10.941680 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39017ed-99c2-4e21-92b0-bc4dfe7aa9d3 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:10.946310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89f990-2325-4bbb-b659-bcd78d7fee6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:10.946977 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89f990-2325-4bbb-b659-bcd78d7fee6c service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:10.947263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89f990-2325-4bbb-b659-bcd78d7fee6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:10.947540 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89f990-2325-4bbb-b659-bcd78d7fee6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:10.949489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ad89f990-2325-4bbb-b659-bcd78d7fee6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:11.020900 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89f990-2325-4bbb-b659-bcd78d7fee6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:11.022396 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89f990-2325-4bbb-b659-bcd78d7fee6c service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:11.032841 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1367/2751] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:20:14.077726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724a495-e8c2-48f7-8c8b-86aa6ef1efbf service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:14.080726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8724a495-e8c2-48f7-8c8b-86aa6ef1efbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:14.109035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724a495-e8c2-48f7-8c8b-86aa6ef1efbf service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:14.116023 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581c562-ed2f-437f-99ea-ad0bc94a6339 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:14.117183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581c562-ed2f-437f-99ea-ad0bc94a6339 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:14.117841 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581c562-ed2f-437f-99ea-ad0bc94a6339 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:14.118670 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581c562-ed2f-437f-99ea-ad0bc94a6339 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:14.122490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a581c562-ed2f-437f-99ea-ad0bc94a6339 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:14.136083 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581c562-ed2f-437f-99ea-ad0bc94a6339 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:14.138445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581c562-ed2f-437f-99ea-ad0bc94a6339 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:14.158279 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1385/2752] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:20:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:20:14.164591 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89f990-2325-4bbb-b659-bcd78d7fee6c service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:14.166696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ad89f990-2325-4bbb-b659-bcd78d7fee6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:14.182972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89f990-2325-4bbb-b659-bcd78d7fee6c service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:14.190543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:14.191234 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:14.191552 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:14.192534 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] PATH_INFO: `/v3/users/b7275026b00b4e93bed7682b3ff2525c` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:14.193831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7275026b00b4e93bed7682b3ff2525c)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:14.197333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:14.252602 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7275026b00b4e93bed7682b3ff2525c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:20:14.253681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7275026b00b4e93bed7682b3ff2525c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:20:14.263845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7275026b00b4e93bed7682b3ff2525c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:20:14.264512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7275026b00b4e93bed7682b3ff2525c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:20:14.272651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-226cb4a7-051c-40f4-80b5-4c292b0d2283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7275026b00b4e93bed7682b3ff2525c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:20:14.293016 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1368/2753] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:20:14 2022] DELETE /identity/v3/users/b7275026b00b4e93bed7682b3ff2525c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:20:14.481393 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581c562-ed2f-437f-99ea-ad0bc94a6339 service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:14.492989 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581c562-ed2f-437f-99ea-ad0bc94a6339 service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:14.556987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:14.557868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:14.558182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:14.558457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] PATH_INFO: `/v3/projects/5778b58ba0bb4bebb334d08971c67323` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:14.559681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5778b58ba0bb4bebb334d08971c67323)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:14.561601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:14.618339 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5778b58ba0bb4bebb334d08971c67323', 'request_id': 'req-fbe22e41-fad1-486b-b21c-559bf2b27818'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:20:14.628651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5778b58ba0bb4bebb334d08971c67323', 'request_id': 'req-fbe22e41-fad1-486b-b21c-559bf2b27818'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:20:14.639964 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5778b58ba0bb4bebb334d08971c67323', 'request_id': 'req-fbe22e41-fad1-486b-b21c-559bf2b27818'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:20:14.655189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] The token cache is being invalidate because project 5778b58ba0bb4bebb334d08971c67323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:20:14.655413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:20:14.656063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fbe22e41-fad1-486b-b21c-559bf2b27818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:20:14.657679 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1386/2754] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:20:14 2022] DELETE /identity/v3/projects/5778b58ba0bb4bebb334d08971c67323 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:20:14.663221 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e92ebd-dfda-424e-8510-dd42e8dd0853 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:14.663511 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e92ebd-dfda-424e-8510-dd42e8dd0853 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:14.663795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e92ebd-dfda-424e-8510-dd42e8dd0853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:14.727875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-27e92ebd-dfda-424e-8510-dd42e8dd0853 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:20:14.729134 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-27e92ebd-dfda-424e-8510-dd42e8dd0853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:14.747380 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:20:14.800148 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1369/2755] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:20:14.810916 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d6f20-492f-452e-91a3-25fb0a4b367b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:14.811262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d6f20-492f-452e-91a3-25fb0a4b367b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:14.811553 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d6f20-492f-452e-91a3-25fb0a4b367b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:14.830465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-384d6f20-492f-452e-91a3-25fb0a4b367b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:20:14.831663 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-384d6f20-492f-452e-91a3-25fb0a4b367b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:14.841337 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1387/2756] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:20:14.845772 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e92ebd-dfda-424e-8510-dd42e8dd0853 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:14.856554 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e92ebd-dfda-424e-8510-dd42e8dd0853 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:14.862184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e7758-5d8f-49c6-866c-9ad0952ce98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:14.862899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e7758-5d8f-49c6-866c-9ad0952ce98f admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:14.863187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e7758-5d8f-49c6-866c-9ad0952ce98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:14.863527 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e7758-5d8f-49c6-866c-9ad0952ce98f admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:14.864277 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e7758-5d8f-49c6-866c-9ad0952ce98f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:20:14.864664 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e7758-5d8f-49c6-866c-9ad0952ce98f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:14.866273 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e7758-5d8f-49c6-866c-9ad0952ce98f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:14.882256 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1370/2757] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:20:14.886952 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d6f20-492f-452e-91a3-25fb0a4b367b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:14.897070 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d6f20-492f-452e-91a3-25fb0a4b367b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:14.902959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691eeb5-3260-4e55-8b92-49b057d8f8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:14.903652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691eeb5-3260-4e55-8b92-49b057d8f8ae admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:14.903940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691eeb5-3260-4e55-8b92-49b057d8f8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:14.904207 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691eeb5-3260-4e55-8b92-49b057d8f8ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:14.904897 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691eeb5-3260-4e55-8b92-49b057d8f8ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:14.906965 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691eeb5-3260-4e55-8b92-49b057d8f8ae admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:14.940761 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1388/2758] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:20:14 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:20:14.945684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e7758-5d8f-49c6-866c-9ad0952ce98f admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:14.960331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e7758-5d8f-49c6-866c-9ad0952ce98f admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:15.025587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedd2f4-8da0-42b6-bdcd-c63b5fa52b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:15.026348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedd2f4-8da0-42b6-bdcd-c63b5fa52b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:15.026637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedd2f4-8da0-42b6-bdcd-c63b5fa52b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:15.026940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedd2f4-8da0-42b6-bdcd-c63b5fa52b09 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:15.027644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedd2f4-8da0-42b6-bdcd-c63b5fa52b09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:15.029594 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedd2f4-8da0-42b6-bdcd-c63b5fa52b09 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:15.054006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1371/2759] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:20:14 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:20:15.059751 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691eeb5-3260-4e55-8b92-49b057d8f8ae admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:15.072225 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691eeb5-3260-4e55-8b92-49b057d8f8ae admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:15.077589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61dbca-f735-416b-adc3-23ec0bdb3819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:15.078295 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61dbca-f735-416b-adc3-23ec0bdb3819 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:15.078870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61dbca-f735-416b-adc3-23ec0bdb3819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:15.078870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61dbca-f735-416b-adc3-23ec0bdb3819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:15.079560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61dbca-f735-416b-adc3-23ec0bdb3819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:15.081165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61dbca-f735-416b-adc3-23ec0bdb3819 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:15.093062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1389/2760] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:20:15.097310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedd2f4-8da0-42b6-bdcd-c63b5fa52b09 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:15.108845 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedd2f4-8da0-42b6-bdcd-c63b5fa52b09 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:15.114556 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb6bed-bd35-4f1c-a896-eabd824a4d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:15.115244 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb6bed-bd35-4f1c-a896-eabd824a4d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:15.115518 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb6bed-bd35-4f1c-a896-eabd824a4d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:15.115793 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb6bed-bd35-4f1c-a896-eabd824a4d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:15.116526 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb6bed-bd35-4f1c-a896-eabd824a4d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:15.118172 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb6bed-bd35-4f1c-a896-eabd824a4d3f admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:15.129248 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1372/2761] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:20:15.133707 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61dbca-f735-416b-adc3-23ec0bdb3819 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:15.144306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61dbca-f735-416b-adc3-23ec0bdb3819 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:15.150360 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498befa-f1f0-4d86-a4c5-6c1c541893fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:15.151299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498befa-f1f0-4d86-a4c5-6c1c541893fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:15.151576 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498befa-f1f0-4d86-a4c5-6c1c541893fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:15.151876 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498befa-f1f0-4d86-a4c5-6c1c541893fb admin admin] PATH_INFO: `/v3/projects/90a3abf875c447d7afbd02d79fad4287/users/fb3a76fedfe349cb919d36ee4d688749/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:15.170621 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498befa-f1f0-4d86-a4c5-6c1c541893fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a3abf875c447d7afbd02d79fad4287, user_id=fb3a76fedfe349cb919d36ee4d688749, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:15.173200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498befa-f1f0-4d86-a4c5-6c1c541893fb admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:15.184476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:20:15.191168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1390/2762] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:20:15 2022] PUT /identity/v3/projects/90a3abf875c447d7afbd02d79fad4287/users/fb3a76fedfe349cb919d36ee4d688749/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:20:15.197163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffbc35-647f-4ee9-b396-1269e23c0d69 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:15.197693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffbc35-647f-4ee9-b396-1269e23c0d69 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:15.197792 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffbc35-647f-4ee9-b396-1269e23c0d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:15.217170 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-87ffbc35-647f-4ee9-b396-1269e23c0d69 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:20:15.218446 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-87ffbc35-647f-4ee9-b396-1269e23c0d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:15.280439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1373/2763] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:20:15.290960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498befa-f1f0-4d86-a4c5-6c1c541893fb admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:15.293108 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8498befa-f1f0-4d86-a4c5-6c1c541893fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:15.355388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498befa-f1f0-4d86-a4c5-6c1c541893fb admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:15.359333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a88d5b-b09d-4dc7-b954-e2e309f20523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:15.359968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a88d5b-b09d-4dc7-b954-e2e309f20523 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:15.360274 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a88d5b-b09d-4dc7-b954-e2e309f20523 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:15.360544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a88d5b-b09d-4dc7-b954-e2e309f20523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:15.370681 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a88d5b-b09d-4dc7-b954-e2e309f20523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:15.372132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a88d5b-b09d-4dc7-b954-e2e309f20523 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:15.383101 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1391/2764] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:20:19.536754 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b936e1-e9c8-4dd3-b120-20b775877642 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:19.537187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b936e1-e9c8-4dd3-b120-20b775877642 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:19.537333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b936e1-e9c8-4dd3-b120-20b775877642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:19.568464 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-64b936e1-e9c8-4dd3-b120-20b775877642 None None] MFA Rules not processed for user `fb3a76fedfe349cb919d36ee4d688749`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:20:19.569813 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-64b936e1-e9c8-4dd3-b120-20b775877642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:19.593535 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:20:19.645765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1374/2765] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:20:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:20:19.660335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a88d5b-b09d-4dc7-b954-e2e309f20523 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:20:19.661941 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-16a88d5b-b09d-4dc7-b954-e2e309f20523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:19.744003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a88d5b-b09d-4dc7-b954-e2e309f20523 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:20:19.750822 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c39a1-37f3-431a-9ba9-d213d4dccbd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:20:19.751511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c39a1-37f3-431a-9ba9-d213d4dccbd3 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:19.751789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c39a1-37f3-431a-9ba9-d213d4dccbd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:19.752049 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c39a1-37f3-431a-9ba9-d213d4dccbd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:19.763021 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c39a1-37f3-431a-9ba9-d213d4dccbd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:20:19.764095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c39a1-37f3-431a-9ba9-d213d4dccbd3 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:20:19.774875 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1392/2766] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:20:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:20:21.263268 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef3aa-8c2b-4d42-88e6-da90550844de None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:20:21.263818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef3aa-8c2b-4d42-88e6-da90550844de None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:20:21.263818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fef3aa-8c2b-4d42-88e6-da90550844de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:20:21.282931 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-30fef3aa-8c2b-4d42-88e6-da90550844de None None] MFA Rules not processed for user `fb3a76fedfe349cb919d36ee4d688749`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:20:21.284208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-30fef3aa-8c2b-4d42-88e6-da90550844de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:20:21.291697 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1375/2767] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:20:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:20:29.769855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=60 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:20:31.277222 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=22 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:21:43.094719 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c39a1-37f3-431a-9ba9-d213d4dccbd3 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:21:43.096511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1f1c39a1-37f3-431a-9ba9-d213d4dccbd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:21:43.114150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c39a1-37f3-431a-9ba9-d213d4dccbd3 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:21:43.119348 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128088b-1bcb-4d49-989a-2849351f71a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:21:43.120231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128088b-1bcb-4d49-989a-2849351f71a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:21:43.120534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128088b-1bcb-4d49-989a-2849351f71a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:21:43.120847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128088b-1bcb-4d49-989a-2849351f71a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:21:43.124078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0128088b-1bcb-4d49-989a-2849351f71a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:21:43.139774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128088b-1bcb-4d49-989a-2849351f71a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:21:43.141523 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128088b-1bcb-4d49-989a-2849351f71a6 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:21:43.152193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1393/2768] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:21:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:21:50.527460 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf238a4-07d4-4955-b7f4-3b3e712a196d None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:21:50.528058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf238a4-07d4-4955-b7f4-3b3e712a196d None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:21:50.528058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf238a4-07d4-4955-b7f4-3b3e712a196d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:21:50.547616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7bf238a4-07d4-4955-b7f4-3b3e712a196d None None] MFA Rules not processed for user `fb3a76fedfe349cb919d36ee4d688749`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:21:50.548817 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7bf238a4-07d4-4955-b7f4-3b3e712a196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:21:50.556536 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1376/2769] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:21:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:21:50.890413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128088b-1bcb-4d49-989a-2849351f71a6 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:21:50.900344 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128088b-1bcb-4d49-989a-2849351f71a6 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:21:50.905012 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b0a1b-59a6-4b3f-b946-d71cace12652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:21:50.905633 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0a1b-59a6-4b3f-b946-d71cace12652 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:21:50.905909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0a1b-59a6-4b3f-b946-d71cace12652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:21:50.906315 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0a1b-59a6-4b3f-b946-d71cace12652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:21:50.908411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-af0b0a1b-59a6-4b3f-b946-d71cace12652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:21:50.919398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b0a1b-59a6-4b3f-b946-d71cace12652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:21:50.920840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b0a1b-59a6-4b3f-b946-d71cace12652 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:21:50.929480 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1394/2770] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:21:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:21:53.726725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf238a4-07d4-4955-b7f4-3b3e712a196d None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:21:53.742586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf238a4-07d4-4955-b7f4-3b3e712a196d None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:21:53.757409 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf31a4-4857-4ff0-8e0d-a9e7b7e87307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:21:53.758435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf31a4-4857-4ff0-8e0d-a9e7b7e87307 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:21:53.758714 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf31a4-4857-4ff0-8e0d-a9e7b7e87307 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:21:53.758988 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf31a4-4857-4ff0-8e0d-a9e7b7e87307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:21:53.760990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3cbf31a4-4857-4ff0-8e0d-a9e7b7e87307 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:21:53.774816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf31a4-4857-4ff0-8e0d-a9e7b7e87307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:21:53.776453 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf31a4-4857-4ff0-8e0d-a9e7b7e87307 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:21:53.788894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1377/2771] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:21:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:21:59.727447 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b0a1b-59a6-4b3f-b946-d71cace12652 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:21:59.742423 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b0a1b-59a6-4b3f-b946-d71cace12652 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:21:59.747951 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e592e8-108c-44bf-90b2-c43454a9b772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:21:59.748602 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e592e8-108c-44bf-90b2-c43454a9b772 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:21:59.748882 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e592e8-108c-44bf-90b2-c43454a9b772 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:21:59.749154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e592e8-108c-44bf-90b2-c43454a9b772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:21:59.750950 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b5e592e8-108c-44bf-90b2-c43454a9b772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:21:59.764337 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e592e8-108c-44bf-90b2-c43454a9b772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:21:59.766008 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e592e8-108c-44bf-90b2-c43454a9b772 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:21:59.778787 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1395/2772] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:21:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:22:03.783932 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=9 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:22:09.773974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=18 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:22:37.337920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf31a4-4857-4ff0-8e0d-a9e7b7e87307 service nova] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:22:37.339611 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3cbf31a4-4857-4ff0-8e0d-a9e7b7e87307 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:22:37.439479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf31a4-4857-4ff0-8e0d-a9e7b7e87307 service nova] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:22:37.445263 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b90ddb-8126-4d34-82e6-e7cc9b13fe71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:22:37.445920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b90ddb-8126-4d34-82e6-e7cc9b13fe71 service placement] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:22:37.446226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b90ddb-8126-4d34-82e6-e7cc9b13fe71 service placement] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:22:37.446489 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b90ddb-8126-4d34-82e6-e7cc9b13fe71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:22:37.448388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c7b90ddb-8126-4d34-82e6-e7cc9b13fe71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:22:37.460363 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b90ddb-8126-4d34-82e6-e7cc9b13fe71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:22:37.461943 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b90ddb-8126-4d34-82e6-e7cc9b13fe71 service placement] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:22:37.471886 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1378/2773] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:22:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:22:47.467544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=32 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:23:20.224357 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591ee14-b933-40f7-a7bd-970574dcf4e0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:23:20.224920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591ee14-b933-40f7-a7bd-970574dcf4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:23:20.224920 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591ee14-b933-40f7-a7bd-970574dcf4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:23:20.250562 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a591ee14-b933-40f7-a7bd-970574dcf4e0 None None] MFA Rules not processed for user `fb3a76fedfe349cb919d36ee4d688749`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:23:20.251795 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a591ee14-b933-40f7-a7bd-970574dcf4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:23:20.259521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1396/2774] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:23:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:23:20.588469 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b90ddb-8126-4d34-82e6-e7cc9b13fe71 service placement] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:23:20.600393 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b90ddb-8126-4d34-82e6-e7cc9b13fe71 service placement] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:23:20.604275 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a93218-1f6d-421a-8494-90111f2780f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:23:20.604944 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a93218-1f6d-421a-8494-90111f2780f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:23:20.605221 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a93218-1f6d-421a-8494-90111f2780f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:23:20.605475 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a93218-1f6d-421a-8494-90111f2780f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:23:20.607351 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-20a93218-1f6d-421a-8494-90111f2780f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:23:20.617987 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a93218-1f6d-421a-8494-90111f2780f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:23:20.619621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a93218-1f6d-421a-8494-90111f2780f3 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:23:20.628956 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1379/2775] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:23:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:23:20.969909 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591ee14-b933-40f7-a7bd-970574dcf4e0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:23:20.971587 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a591ee14-b933-40f7-a7bd-970574dcf4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:23:20.988214 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591ee14-b933-40f7-a7bd-970574dcf4e0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:23:20.993465 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2a3c0-b84a-476e-9312-39691a5e31e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:23:20.994150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2a3c0-b84a-476e-9312-39691a5e31e2 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:23:20.994652 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2a3c0-b84a-476e-9312-39691a5e31e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:23:20.994930 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2a3c0-b84a-476e-9312-39691a5e31e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:23:20.996864 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5cf2a3c0-b84a-476e-9312-39691a5e31e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:23:21.009834 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2a3c0-b84a-476e-9312-39691a5e31e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:23:21.011472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2a3c0-b84a-476e-9312-39691a5e31e2 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:23:21.025194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1397/2776] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:23:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:23:30.624913 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=6 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:23:31.019149 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=9 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:24:52.469662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2fee1-7eac-4cec-a8f3-9578fe0c9932 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:24:52.470325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2fee1-7eac-4cec-a8f3-9578fe0c9932 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:24:52.470325 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2fee1-7eac-4cec-a8f3-9578fe0c9932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:24:52.488722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-24b2fee1-7eac-4cec-a8f3-9578fe0c9932 None None] MFA Rules not processed for user `fb3a76fedfe349cb919d36ee4d688749`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:24:52.489975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-24b2fee1-7eac-4cec-a8f3-9578fe0c9932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:24:52.499009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1380/2777] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:24:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:25:02.482944 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:08.469318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2a3c0-b84a-476e-9312-39691a5e31e2 service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:08.470933 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5cf2a3c0-b84a-476e-9312-39691a5e31e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:08.488628 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2a3c0-b84a-476e-9312-39691a5e31e2 service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:08.495141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6629fa6-34e6-4563-8cad-d4ca2e8f79d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:08.496000 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6629fa6-34e6-4563-8cad-d4ca2e8f79d0 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:08.496276 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6629fa6-34e6-4563-8cad-d4ca2e8f79d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:08.496618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6629fa6-34e6-4563-8cad-d4ca2e8f79d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:08.504850 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6629fa6-34e6-4563-8cad-d4ca2e8f79d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:08.506472 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6629fa6-34e6-4563-8cad-d4ca2e8f79d0 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:08.515747 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1398/2778] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:09.967278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2fee1-7eac-4cec-a8f3-9578fe0c9932 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:09.969326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-24b2fee1-7eac-4cec-a8f3-9578fe0c9932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:10.064296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2fee1-7eac-4cec-a8f3-9578fe0c9932 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:10.068335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43924b-c432-469f-bd57-b44cfad07746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:10.068976 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43924b-c432-469f-bd57-b44cfad07746 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:10.069249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43924b-c432-469f-bd57-b44cfad07746 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:10.069550 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43924b-c432-469f-bd57-b44cfad07746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:10.071395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dc43924b-c432-469f-bd57-b44cfad07746 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:10.085494 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43924b-c432-469f-bd57-b44cfad07746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:10.086963 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43924b-c432-469f-bd57-b44cfad07746 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:10.098750 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1381/2779] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:16.968194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6629fa6-34e6-4563-8cad-d4ca2e8f79d0 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:16.983571 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6629fa6-34e6-4563-8cad-d4ca2e8f79d0 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:16.988568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57836d53-d693-4ed8-926e-c3159330751f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:16.989205 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57836d53-d693-4ed8-926e-c3159330751f service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:16.989467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57836d53-d693-4ed8-926e-c3159330751f service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:16.989733 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57836d53-d693-4ed8-926e-c3159330751f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:16.991887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-57836d53-d693-4ed8-926e-c3159330751f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:17.007003 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57836d53-d693-4ed8-926e-c3159330751f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:17.008399 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57836d53-d693-4ed8-926e-c3159330751f service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:17.020500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1399/2780] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:20.093818 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=35 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:26.517693 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43924b-c432-469f-bd57-b44cfad07746 service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:26.528571 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43924b-c432-469f-bd57-b44cfad07746 service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:26.533498 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbecb3-2bda-40c8-b2dd-3f3109882a58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:26.534462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbecb3-2bda-40c8-b2dd-3f3109882a58 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:26.534758 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbecb3-2bda-40c8-b2dd-3f3109882a58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:26.535081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbecb3-2bda-40c8-b2dd-3f3109882a58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:26.550030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbecb3-2bda-40c8-b2dd-3f3109882a58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:26.552058 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbecb3-2bda-40c8-b2dd-3f3109882a58 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:26.562360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1382/2781] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:27.014596 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=12 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:29.557601 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57836d53-d693-4ed8-926e-c3159330751f service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:29.570013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57836d53-d693-4ed8-926e-c3159330751f service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:29.575400 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1aab9e-201b-473e-80ba-de0f41015364 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:29.576053 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aab9e-201b-473e-80ba-de0f41015364 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:29.576329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aab9e-201b-473e-80ba-de0f41015364 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:29.576618 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aab9e-201b-473e-80ba-de0f41015364 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:29.578418 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-df1aab9e-201b-473e-80ba-de0f41015364 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:29.590899 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1aab9e-201b-473e-80ba-de0f41015364 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:29.592537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1aab9e-201b-473e-80ba-de0f41015364 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:29.606593 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1400/2782] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:29.627154 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbecb3-2bda-40c8-b2dd-3f3109882a58 service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:29.639549 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbecb3-2bda-40c8-b2dd-3f3109882a58 service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:29.646915 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:29.647838 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:29.648121 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:29.648449 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] PATH_INFO: `/v3/users/fb3a76fedfe349cb919d36ee4d688749` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:29.649658 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3a76fedfe349cb919d36ee4d688749)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:29.651805 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:29.708601 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3a76fedfe349cb919d36ee4d688749'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:29.709404 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3a76fedfe349cb919d36ee4d688749'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:29.718533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3a76fedfe349cb919d36ee4d688749'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:29.719283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3a76fedfe349cb919d36ee4d688749'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:29.726823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-2a023c31-ab5b-48c6-bb52-e781bc92efcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3a76fedfe349cb919d36ee4d688749'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:29.747614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1383/2783] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:26:29 2022] DELETE /identity/v3/users/fb3a76fedfe349cb919d36ee4d688749 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:29.934844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1aab9e-201b-473e-80ba-de0f41015364 service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:29.946303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1aab9e-201b-473e-80ba-de0f41015364 service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:30.007441 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:30.008253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.008534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.008833 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] PATH_INFO: `/v3/projects/90a3abf875c447d7afbd02d79fad4287` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.010055 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a3abf875c447d7afbd02d79fad4287)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:30.012011 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:30.059790 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a3abf875c447d7afbd02d79fad4287', 'request_id': 'req-05416763-553b-45ee-9e01-4807ec8817e6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:30.067458 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a3abf875c447d7afbd02d79fad4287', 'request_id': 'req-05416763-553b-45ee-9e01-4807ec8817e6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:30.072991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a3abf875c447d7afbd02d79fad4287', 'request_id': 'req-05416763-553b-45ee-9e01-4807ec8817e6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:30.084536 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] The token cache is being invalidate because project 90a3abf875c447d7afbd02d79fad4287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:26:30.084826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:30.085471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05416763-553b-45ee-9e01-4807ec8817e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:30.087065 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1401/2784] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:26:29 2022] DELETE /identity/v3/projects/90a3abf875c447d7afbd02d79fad4287 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:30.092318 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f1629-974a-456c-9f22-49b02b01cb89 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.092633 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f1629-974a-456c-9f22-49b02b01cb89 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.092899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f1629-974a-456c-9f22-49b02b01cb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.111996 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-dc9f1629-974a-456c-9f22-49b02b01cb89 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:30.113515 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-dc9f1629-974a-456c-9f22-49b02b01cb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:30.177587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1384/2785] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:30.185664 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5838f3-05ec-40c1-9d12-2cf6b8f6771b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.185956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5838f3-05ec-40c1-9d12-2cf6b8f6771b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.186275 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5838f3-05ec-40c1-9d12-2cf6b8f6771b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.205458 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1e5838f3-05ec-40c1-9d12-2cf6b8f6771b None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:30.206699 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1e5838f3-05ec-40c1-9d12-2cf6b8f6771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:30.215987 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1402/2786] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:30.220816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f1629-974a-456c-9f22-49b02b01cb89 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:30.235890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f1629-974a-456c-9f22-49b02b01cb89 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:30.242816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9a7ce-5376-417f-b76d-60f9eb2182ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:30.243512 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9a7ce-5376-417f-b76d-60f9eb2182ca admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.243823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9a7ce-5376-417f-b76d-60f9eb2182ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.244078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9a7ce-5376-417f-b76d-60f9eb2182ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.244814 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9a7ce-5376-417f-b76d-60f9eb2182ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:26:30.245156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9a7ce-5376-417f-b76d-60f9eb2182ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:30.246807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9a7ce-5376-417f-b76d-60f9eb2182ca admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:30.259320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1385/2787] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:26:30.265801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5838f3-05ec-40c1-9d12-2cf6b8f6771b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:30.276337 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5838f3-05ec-40c1-9d12-2cf6b8f6771b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:30.282646 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0b93f-ac6d-4309-9796-e5e711fb8ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:30.283304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0b93f-ac6d-4309-9796-e5e711fb8ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.283589 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0b93f-ac6d-4309-9796-e5e711fb8ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.283826 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0b93f-ac6d-4309-9796-e5e711fb8ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.284528 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0b93f-ac6d-4309-9796-e5e711fb8ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:30.286503 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0b93f-ac6d-4309-9796-e5e711fb8ce3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:30.328854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1403/2788] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:26:30 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:26:30.333322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9a7ce-5376-417f-b76d-60f9eb2182ca admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:30.344291 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9a7ce-5376-417f-b76d-60f9eb2182ca admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:30.391482 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:30.408973 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29833f-7d9b-430a-b8b4-7019808487b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:30.409774 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29833f-7d9b-430a-b8b4-7019808487b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.410052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29833f-7d9b-430a-b8b4-7019808487b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.410359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29833f-7d9b-430a-b8b4-7019808487b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.411056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29833f-7d9b-430a-b8b4-7019808487b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:30.413703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29833f-7d9b-430a-b8b4-7019808487b8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:30.439159 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1386/2789] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:26:30 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:26:30.444530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0b93f-ac6d-4309-9796-e5e711fb8ce3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:30.460333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0b93f-ac6d-4309-9796-e5e711fb8ce3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:30.465455 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbb327-6da4-4b19-b9af-b3efd2616eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:30.466291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbb327-6da4-4b19-b9af-b3efd2616eef admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.466639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbb327-6da4-4b19-b9af-b3efd2616eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.466978 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbb327-6da4-4b19-b9af-b3efd2616eef admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.467711 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cbb327-6da4-4b19-b9af-b3efd2616eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:30.469351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cbb327-6da4-4b19-b9af-b3efd2616eef admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:30.480615 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1404/2790] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:26:30.484888 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29833f-7d9b-430a-b8b4-7019808487b8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:30.496504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29833f-7d9b-430a-b8b4-7019808487b8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:30.504090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7e642-4bd6-4bcb-8589-2a92bd7732f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:30.505090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7e642-4bd6-4bcb-8589-2a92bd7732f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.505509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7e642-4bd6-4bcb-8589-2a92bd7732f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.505804 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7e642-4bd6-4bcb-8589-2a92bd7732f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.506798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7e642-4bd6-4bcb-8589-2a92bd7732f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:30.508443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7e642-4bd6-4bcb-8589-2a92bd7732f9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:30.518866 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1387/2791] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:26:30.523301 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbb327-6da4-4b19-b9af-b3efd2616eef admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:30.533505 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cbb327-6da4-4b19-b9af-b3efd2616eef admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:30.539222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf8066-6b74-4835-9d96-32ae4f10684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:30.540115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf8066-6b74-4835-9d96-32ae4f10684d admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.540397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf8066-6b74-4835-9d96-32ae4f10684d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.540677 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf8066-6b74-4835-9d96-32ae4f10684d admin admin] PATH_INFO: `/v3/projects/28903e6e969a429b98b707f6db55c27c/users/497cac9e38c241efa5199042a38d212b/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.556240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf8066-6b74-4835-9d96-32ae4f10684d admin admin] RBAC: Authorizing `identity:create_grant(project_id=28903e6e969a429b98b707f6db55c27c, user_id=497cac9e38c241efa5199042a38d212b, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:30.558953 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf8066-6b74-4835-9d96-32ae4f10684d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:30.573938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1405/2792] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:26:30 2022] PUT /identity/v3/projects/28903e6e969a429b98b707f6db55c27c/users/497cac9e38c241efa5199042a38d212b/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:30.584906 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509f14e-378d-4169-9ed4-0721ab8f84ba None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.585095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509f14e-378d-4169-9ed4-0721ab8f84ba None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.585360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509f14e-378d-4169-9ed4-0721ab8f84ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.613749 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-7509f14e-378d-4169-9ed4-0721ab8f84ba None None] MFA Rules not processed for user `497cac9e38c241efa5199042a38d212b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:30.615083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-7509f14e-378d-4169-9ed4-0721ab8f84ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:30.687330 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1388/2793] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:30.701770 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf8066-6b74-4835-9d96-32ae4f10684d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:30.704516 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d6cf8066-6b74-4835-9d96-32ae4f10684d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:30.714877 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:30.797873 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf8066-6b74-4835-9d96-32ae4f10684d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:30.808329 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82f795-9f94-4e3b-ad51-08618f22c749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:30.809278 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82f795-9f94-4e3b-ad51-08618f22c749 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:30.809660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82f795-9f94-4e3b-ad51-08618f22c749 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:30.810094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82f795-9f94-4e3b-ad51-08618f22c749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:30.821372 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82f795-9f94-4e3b-ad51-08618f22c749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:30.824162 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82f795-9f94-4e3b-ad51-08618f22c749 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:30.832609 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1406/2794] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:31.057052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509f14e-378d-4169-9ed4-0721ab8f84ba None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:31.128332 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509f14e-378d-4169-9ed4-0721ab8f84ba None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:31.133880 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:31.134875 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:31.135178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:31.135451 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] PATH_INFO: `/v3/users/497cac9e38c241efa5199042a38d212b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:31.136699 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=497cac9e38c241efa5199042a38d212b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:31.139292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:31.198502 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497cac9e38c241efa5199042a38d212b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:31.199252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497cac9e38c241efa5199042a38d212b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:31.209190 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497cac9e38c241efa5199042a38d212b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:31.210030 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497cac9e38c241efa5199042a38d212b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:31.212230 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:31.218675 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497cac9e38c241efa5199042a38d212b'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:31.237187 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1389/2795] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:26:31 2022] DELETE /identity/v3/users/497cac9e38c241efa5199042a38d212b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:31.242118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a44b4-89ab-4568-935c-4c85286859bb None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:31.242424 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a44b4-89ab-4568-935c-4c85286859bb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:31.242699 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a44b4-89ab-4568-935c-4c85286859bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:31.263687 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c64a44b4-89ab-4568-935c-4c85286859bb None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:31.264919 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c64a44b4-89ab-4568-935c-4c85286859bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:31.336034 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1407/2796] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:31.345957 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:31.347476 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:31.404732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ecf35a-d1ca-4bbd-a123-6c14a747c9cf admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:31.409063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a562d-273f-4a19-85b3-4e1cc7282b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:31.409757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a562d-273f-4a19-85b3-4e1cc7282b72 service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:31.410019 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a562d-273f-4a19-85b3-4e1cc7282b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:31.410335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a562d-273f-4a19-85b3-4e1cc7282b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:31.420286 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a562d-273f-4a19-85b3-4e1cc7282b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:31.421760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a562d-273f-4a19-85b3-4e1cc7282b72 service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:31.432459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1390/2797] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:31.731157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a44b4-89ab-4568-935c-4c85286859bb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:31.732743 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c64a44b4-89ab-4568-935c-4c85286859bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:31.747113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a44b4-89ab-4568-935c-4c85286859bb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:31.752335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:31.753248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:31.753497 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:31.753771 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] PATH_INFO: `/v3/projects/28903e6e969a429b98b707f6db55c27c` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:31.755253 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28903e6e969a429b98b707f6db55c27c)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:31.757501 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:31.806678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28903e6e969a429b98b707f6db55c27c', 'request_id': 'req-5e7d98b9-f38b-4df3-9556-da86bb173e08'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:31.814639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28903e6e969a429b98b707f6db55c27c', 'request_id': 'req-5e7d98b9-f38b-4df3-9556-da86bb173e08'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:31.820289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28903e6e969a429b98b707f6db55c27c', 'request_id': 'req-5e7d98b9-f38b-4df3-9556-da86bb173e08'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:31.831312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] The token cache is being invalidate because project 28903e6e969a429b98b707f6db55c27c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:26:31.831638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:31.832318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e7d98b9-f38b-4df3-9556-da86bb173e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:31.833931 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1408/2798] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:26:31 2022] DELETE /identity/v3/projects/28903e6e969a429b98b707f6db55c27c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:31.839440 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c025f1-5e3e-4023-9bbb-34cc8ad44627 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:31.839920 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c025f1-5e3e-4023-9bbb-34cc8ad44627 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:31.840043 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c025f1-5e3e-4023-9bbb-34cc8ad44627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:31.859039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-f8c025f1-5e3e-4023-9bbb-34cc8ad44627 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:31.860278 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-f8c025f1-5e3e-4023-9bbb-34cc8ad44627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:31.925074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1391/2799] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:31.933430 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a5466-5b67-4ff1-8f18-bc05d1112f93 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:31.933726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a5466-5b67-4ff1-8f18-bc05d1112f93 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:31.934010 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a5466-5b67-4ff1-8f18-bc05d1112f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:31.953999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c22a5466-5b67-4ff1-8f18-bc05d1112f93 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:31.955240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c22a5466-5b67-4ff1-8f18-bc05d1112f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:31.968173 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1409/2800] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:31.973398 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c025f1-5e3e-4023-9bbb-34cc8ad44627 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:31.984822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c025f1-5e3e-4023-9bbb-34cc8ad44627 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:31.992141 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c5974-51c6-4d55-aeab-9f3b75752d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:31.993068 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c5974-51c6-4d55-aeab-9f3b75752d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:31.993419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c5974-51c6-4d55-aeab-9f3b75752d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:31.993800 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c5974-51c6-4d55-aeab-9f3b75752d59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:31.994980 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c5974-51c6-4d55-aeab-9f3b75752d59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:26:31.995421 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c5974-51c6-4d55-aeab-9f3b75752d59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:31.997892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c5974-51c6-4d55-aeab-9f3b75752d59 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.014060 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1392/2801] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:26:32.018612 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a5466-5b67-4ff1-8f18-bc05d1112f93 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.028847 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a5466-5b67-4ff1-8f18-bc05d1112f93 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.034684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b536230-82a8-4d58-bd53-a34e3411529a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.035364 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b536230-82a8-4d58-bd53-a34e3411529a admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.035640 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b536230-82a8-4d58-bd53-a34e3411529a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.035917 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b536230-82a8-4d58-bd53-a34e3411529a admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.036613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b536230-82a8-4d58-bd53-a34e3411529a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.039046 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b536230-82a8-4d58-bd53-a34e3411529a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.042678 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:32.071426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1410/2802] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:26:32 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:26:32.075746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c5974-51c6-4d55-aeab-9f3b75752d59 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.086179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c5974-51c6-4d55-aeab-9f3b75752d59 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.144420 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a0956-0b94-47d1-b636-5ca823cf1ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.145084 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a0956-0b94-47d1-b636-5ca823cf1ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.145362 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a0956-0b94-47d1-b636-5ca823cf1ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.145638 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a0956-0b94-47d1-b636-5ca823cf1ba9 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.146349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a0956-0b94-47d1-b636-5ca823cf1ba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.148776 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a0956-0b94-47d1-b636-5ca823cf1ba9 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.158078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:32.172459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1393/2803] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:26:32 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:26:32.177048 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b536230-82a8-4d58-bd53-a34e3411529a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.188115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b536230-82a8-4d58-bd53-a34e3411529a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.193191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbdf6a-7167-4f68-900b-b1f091cd49a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.194019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbdf6a-7167-4f68-900b-b1f091cd49a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.194566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbdf6a-7167-4f68-900b-b1f091cd49a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.194898 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbdf6a-7167-4f68-900b-b1f091cd49a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.195645 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dbdf6a-7167-4f68-900b-b1f091cd49a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.197272 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dbdf6a-7167-4f68-900b-b1f091cd49a9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.208109 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1411/2804] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:26:32.212429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a0956-0b94-47d1-b636-5ca823cf1ba9 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.222760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a0956-0b94-47d1-b636-5ca823cf1ba9 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.229726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8ed68-e84c-45fc-8efd-e7d820b81167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.230428 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8ed68-e84c-45fc-8efd-e7d820b81167 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.230705 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8ed68-e84c-45fc-8efd-e7d820b81167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.230975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8ed68-e84c-45fc-8efd-e7d820b81167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.231704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8ed68-e84c-45fc-8efd-e7d820b81167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.233322 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8ed68-e84c-45fc-8efd-e7d820b81167 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.246828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1394/2805] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:26:32.251731 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbdf6a-7167-4f68-900b-b1f091cd49a9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.264153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbdf6a-7167-4f68-900b-b1f091cd49a9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.269384 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c3996-eb89-48e3-975e-937c679a4034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.270341 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c3996-eb89-48e3-975e-937c679a4034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.270625 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c3996-eb89-48e3-975e-937c679a4034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.270904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c3996-eb89-48e3-975e-937c679a4034 admin admin] PATH_INFO: `/v3/projects/966d3d632e804f35a3f1aabac2c813a6/users/9193fd0568374c2c86961fe531027862/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.286165 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c3996-eb89-48e3-975e-937c679a4034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966d3d632e804f35a3f1aabac2c813a6, user_id=9193fd0568374c2c86961fe531027862, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.288888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c3996-eb89-48e3-975e-937c679a4034 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.304365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1412/2806] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:26:32 2022] PUT /identity/v3/projects/966d3d632e804f35a3f1aabac2c813a6/users/9193fd0568374c2c86961fe531027862/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:32.315104 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e078a-7d75-48cd-a32e-4fce9c7bc37e None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.315388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e078a-7d75-48cd-a32e-4fce9c7bc37e None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.315663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e078a-7d75-48cd-a32e-4fce9c7bc37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.347604 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-fd1e078a-7d75-48cd-a32e-4fce9c7bc37e None None] MFA Rules not processed for user `9193fd0568374c2c86961fe531027862`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:32.348865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-fd1e078a-7d75-48cd-a32e-4fce9c7bc37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:32.416637 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1395/2807] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:32.424304 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c3996-eb89-48e3-975e-937c679a4034 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.504013 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c3996-eb89-48e3-975e-937c679a4034 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.510352 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a1e7b-0526-4064-8771-2e6b6327e758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.511323 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a1e7b-0526-4064-8771-2e6b6327e758 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.511735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a1e7b-0526-4064-8771-2e6b6327e758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.512140 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a1e7b-0526-4064-8771-2e6b6327e758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.513215 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a1e7b-0526-4064-8771-2e6b6327e758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.516194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a1e7b-0526-4064-8771-2e6b6327e758 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.547037 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1413/2808] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:26:32 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:26:32.551798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e078a-7d75-48cd-a32e-4fce9c7bc37e None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.563386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e078a-7d75-48cd-a32e-4fce9c7bc37e None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.622847 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84161c-f192-4637-919d-ad805dbea53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.623522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84161c-f192-4637-919d-ad805dbea53a admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.623803 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84161c-f192-4637-919d-ad805dbea53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.624074 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84161c-f192-4637-919d-ad805dbea53a admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.624769 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84161c-f192-4637-919d-ad805dbea53a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.626742 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84161c-f192-4637-919d-ad805dbea53a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.647123 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1396/2809] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:26:32 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:26:32.651606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a1e7b-0526-4064-8771-2e6b6327e758 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.662489 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a1e7b-0526-4064-8771-2e6b6327e758 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.668161 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ef0cf-bba7-41e3-a112-ebd3788be0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.668829 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ef0cf-bba7-41e3-a112-ebd3788be0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.669124 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ef0cf-bba7-41e3-a112-ebd3788be0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.669385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ef0cf-bba7-41e3-a112-ebd3788be0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.670114 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ef0cf-bba7-41e3-a112-ebd3788be0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.671746 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ef0cf-bba7-41e3-a112-ebd3788be0f9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.683521 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1414/2810] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:26:32.688026 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84161c-f192-4637-919d-ad805dbea53a admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.699207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84161c-f192-4637-919d-ad805dbea53a admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.704737 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c32d1-5724-4414-9293-090ee3466639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.705636 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c32d1-5724-4414-9293-090ee3466639 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.705924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c32d1-5724-4414-9293-090ee3466639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.706233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c32d1-5724-4414-9293-090ee3466639 admin admin] PATH_INFO: `/v3/projects/8c79863ee8ea498bb6d7e7efe9a9004b/users/fad5b83bd6f44d729ed9c466ec48d8b7/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.721639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c32d1-5724-4414-9293-090ee3466639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c79863ee8ea498bb6d7e7efe9a9004b, user_id=fad5b83bd6f44d729ed9c466ec48d8b7, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.724362 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c32d1-5724-4414-9293-090ee3466639 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.739660 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1397/2811] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:26:32 2022] PUT /identity/v3/projects/8c79863ee8ea498bb6d7e7efe9a9004b/users/fad5b83bd6f44d729ed9c466ec48d8b7/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:32.744519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ef0cf-bba7-41e3-a112-ebd3788be0f9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.759146 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ef0cf-bba7-41e3-a112-ebd3788be0f9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.819044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807bb4a-f65c-457f-9729-4b40f6e9e7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.820240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807bb4a-f65c-457f-9729-4b40f6e9e7de admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.820522 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807bb4a-f65c-457f-9729-4b40f6e9e7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.820796 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807bb4a-f65c-457f-9729-4b40f6e9e7de admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.821514 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807bb4a-f65c-457f-9729-4b40f6e9e7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.823154 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807bb4a-f65c-457f-9729-4b40f6e9e7de admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.833240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1415/2812] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:26:32.838050 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c32d1-5724-4414-9293-090ee3466639 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:32.849639 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c32d1-5724-4414-9293-090ee3466639 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:32.855303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20505ea7-9398-46f2-a30d-1532bb62d39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:32.856205 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505ea7-9398-46f2-a30d-1532bb62d39b admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.856523 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505ea7-9398-46f2-a30d-1532bb62d39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.856785 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505ea7-9398-46f2-a30d-1532bb62d39b admin admin] PATH_INFO: `/v3/projects/8c79863ee8ea498bb6d7e7efe9a9004b/users/fad5b83bd6f44d729ed9c466ec48d8b7/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.858898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20505ea7-9398-46f2-a30d-1532bb62d39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c79863ee8ea498bb6d7e7efe9a9004b, user_id=fad5b83bd6f44d729ed9c466ec48d8b7, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:32.861546 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20505ea7-9398-46f2-a30d-1532bb62d39b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:32.872392 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1398/2813] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:26:32 2022] PUT /identity/v3/projects/8c79863ee8ea498bb6d7e7efe9a9004b/users/fad5b83bd6f44d729ed9c466ec48d8b7/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:32.936182 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea118c2-1fc4-47de-a18d-55eb44c937c4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:32.936439 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea118c2-1fc4-47de-a18d-55eb44c937c4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:32.936755 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea118c2-1fc4-47de-a18d-55eb44c937c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:32.965014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4ea118c2-1fc4-47de-a18d-55eb44c937c4 None None] MFA Rules not processed for user `fad5b83bd6f44d729ed9c466ec48d8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:32.966251 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4ea118c2-1fc4-47de-a18d-55eb44c937c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:32.974824 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:33.037132 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1416/2814] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:33.049850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20505ea7-9398-46f2-a30d-1532bb62d39b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:33.051735 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-20505ea7-9398-46f2-a30d-1532bb62d39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:33.081708 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:33.114422 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20505ea7-9398-46f2-a30d-1532bb62d39b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:33.118760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5f913-6a15-45a1-8e78-30e7508fd834 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:33.119401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5f913-6a15-45a1-8e78-30e7508fd834 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:33.119681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5f913-6a15-45a1-8e78-30e7508fd834 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:33.119950 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5f913-6a15-45a1-8e78-30e7508fd834 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:33.129265 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5f913-6a15-45a1-8e78-30e7508fd834 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:33.130752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5f913-6a15-45a1-8e78-30e7508fd834 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:33.141015 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1399/2815] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:33.214511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea118c2-1fc4-47de-a18d-55eb44c937c4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:33.284099 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea118c2-1fc4-47de-a18d-55eb44c937c4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:33.289231 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:33.290042 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:33.290356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:33.290636 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] PATH_INFO: `/v3/users/9193fd0568374c2c86961fe531027862` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:33.293242 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9193fd0568374c2c86961fe531027862)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:33.295200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:33.347223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9193fd0568374c2c86961fe531027862'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:33.358464 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9193fd0568374c2c86961fe531027862'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:33.359454 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9193fd0568374c2c86961fe531027862'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:33.360453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9193fd0568374c2c86961fe531027862'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:33.375098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9193fd0568374c2c86961fe531027862'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:33.387797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1417/2816] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:26:33 2022] DELETE /identity/v3/users/9193fd0568374c2c86961fe531027862 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:33.392414 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286f02-d409-4173-b401-665aad5f1199 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:33.392696 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286f02-d409-4173-b401-665aad5f1199 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:33.392972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286f02-d409-4173-b401-665aad5f1199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:33.411175 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-df286f02-d409-4173-b401-665aad5f1199 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:33.412379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-df286f02-d409-4173-b401-665aad5f1199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:33.473479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1400/2817] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:33.484412 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:33.485947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:33.516660 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:33.548157 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d704f3-9a33-4e80-86df-c9a1138e2c1a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:33.552445 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520bf10-a03c-4d31-a075-a62bb5321c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:33.553093 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520bf10-a03c-4d31-a075-a62bb5321c92 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:33.553379 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520bf10-a03c-4d31-a075-a62bb5321c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:33.553765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520bf10-a03c-4d31-a075-a62bb5321c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:33.563480 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520bf10-a03c-4d31-a075-a62bb5321c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:33.564923 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520bf10-a03c-4d31-a075-a62bb5321c92 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:33.574901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1418/2818] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:33.857892 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df286f02-d409-4173-b401-665aad5f1199 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:33.859399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-df286f02-d409-4173-b401-665aad5f1199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:33.875226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df286f02-d409-4173-b401-665aad5f1199 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:33.881734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:33.882558 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:33.882838 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:33.883131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] PATH_INFO: `/v3/projects/966d3d632e804f35a3f1aabac2c813a6` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:33.884328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=966d3d632e804f35a3f1aabac2c813a6)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:33.886290 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:33.937734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966d3d632e804f35a3f1aabac2c813a6', 'request_id': 'req-da831ed6-6ba7-4c5a-a559-ca5b52550e27'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:33.947349 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966d3d632e804f35a3f1aabac2c813a6', 'request_id': 'req-da831ed6-6ba7-4c5a-a559-ca5b52550e27'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:33.953551 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966d3d632e804f35a3f1aabac2c813a6', 'request_id': 'req-da831ed6-6ba7-4c5a-a559-ca5b52550e27'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:33.965196 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] The token cache is being invalidate because project 966d3d632e804f35a3f1aabac2c813a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:26:33.965480 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:33.966146 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:33.967733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1401/2819] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:26:33 2022] DELETE /identity/v3/projects/966d3d632e804f35a3f1aabac2c813a6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:33.971956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520bf10-a03c-4d31-a075-a62bb5321c92 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:33.982382 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520bf10-a03c-4d31-a075-a62bb5321c92 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:34.040583 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:34.041470 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:34.041752 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:34.042029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] PATH_INFO: `/v3/users/fad5b83bd6f44d729ed9c466ec48d8b7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:34.043241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad5b83bd6f44d729ed9c466ec48d8b7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:34.045321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:34.098166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad5b83bd6f44d729ed9c466ec48d8b7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:34.106762 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad5b83bd6f44d729ed9c466ec48d8b7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:34.107328 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad5b83bd6f44d729ed9c466ec48d8b7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:34.107991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad5b83bd6f44d729ed9c466ec48d8b7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:34.120390 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a7e53c72-5c50-490c-95ec-d5600aca70b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad5b83bd6f44d729ed9c466ec48d8b7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:34.132672 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1419/2820] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:26:33 2022] DELETE /identity/v3/users/fad5b83bd6f44d729ed9c466ec48d8b7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:34.432853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:34.434565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:34.450354 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da831ed6-6ba7-4c5a-a559-ca5b52550e27 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:34.498779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:34.510563 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:34.511522 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:34.511777 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:34.512007 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] PATH_INFO: `/v3/projects/8c79863ee8ea498bb6d7e7efe9a9004b` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:34.513216 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c79863ee8ea498bb6d7e7efe9a9004b)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:34.515173 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:34.567614 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c79863ee8ea498bb6d7e7efe9a9004b', 'request_id': 'req-68c962d3-5087-4d66-b258-c5601286804a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:34.577772 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c79863ee8ea498bb6d7e7efe9a9004b', 'request_id': 'req-68c962d3-5087-4d66-b258-c5601286804a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:34.583429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c79863ee8ea498bb6d7e7efe9a9004b', 'request_id': 'req-68c962d3-5087-4d66-b258-c5601286804a'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:34.594981 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] The token cache is being invalidate because project 8c79863ee8ea498bb6d7e7efe9a9004b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:26:34.595269 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:34.595898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-68c962d3-5087-4d66-b258-c5601286804a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:34.597462 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1402/2821] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:26:34 2022] DELETE /identity/v3/projects/8c79863ee8ea498bb6d7e7efe9a9004b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:34.607325 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d92e5-4998-4783-baa9-3916fb4ce05e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:34.607597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d92e5-4998-4783-baa9-3916fb4ce05e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:34.607870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d92e5-4998-4783-baa9-3916fb4ce05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:34.628358 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2b3d92e5-4998-4783-baa9-3916fb4ce05e None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:34.629485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2b3d92e5-4998-4783-baa9-3916fb4ce05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:34.675241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:34.694544 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1420/2822] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:34.702872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb6400-5a6f-48df-b28b-4517a52271a2 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:34.703172 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb6400-5a6f-48df-b28b-4517a52271a2 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:34.703458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb6400-5a6f-48df-b28b-4517a52271a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:34.724010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-83eb6400-5a6f-48df-b28b-4517a52271a2 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:34.725215 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-83eb6400-5a6f-48df-b28b-4517a52271a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:34.733929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1403/2823] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:34.738853 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d92e5-4998-4783-baa9-3916fb4ce05e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:34.750036 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d92e5-4998-4783-baa9-3916fb4ce05e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:34.755995 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ca6f-beb0-40d3-81bf-00b2b82db948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:34.756651 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ca6f-beb0-40d3-81bf-00b2b82db948 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:34.756934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ca6f-beb0-40d3-81bf-00b2b82db948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:34.757208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ca6f-beb0-40d3-81bf-00b2b82db948 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:34.757942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ca6f-beb0-40d3-81bf-00b2b82db948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:26:34.758277 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ca6f-beb0-40d3-81bf-00b2b82db948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:34.759888 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ca6f-beb0-40d3-81bf-00b2b82db948 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:34.771938 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1421/2824] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:26:34.777178 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb6400-5a6f-48df-b28b-4517a52271a2 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:34.792162 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb6400-5a6f-48df-b28b-4517a52271a2 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:34.801203 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891209e-0bc9-4772-803a-f99e69eed463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:34.802149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891209e-0bc9-4772-803a-f99e69eed463 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:34.802433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891209e-0bc9-4772-803a-f99e69eed463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:34.802703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8891209e-0bc9-4772-803a-f99e69eed463 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:34.803423 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891209e-0bc9-4772-803a-f99e69eed463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:34.805375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8891209e-0bc9-4772-803a-f99e69eed463 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:34.834561 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1404/2825] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:26:34 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:26:34.839095 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ca6f-beb0-40d3-81bf-00b2b82db948 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:34.849427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ca6f-beb0-40d3-81bf-00b2b82db948 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:34.908045 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a67c78-def0-4409-9bfb-b33c122a737d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:34.908607 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a67c78-def0-4409-9bfb-b33c122a737d admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:34.908899 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a67c78-def0-4409-9bfb-b33c122a737d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:34.909152 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a67c78-def0-4409-9bfb-b33c122a737d admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:34.910391 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a67c78-def0-4409-9bfb-b33c122a737d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:34.911840 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a67c78-def0-4409-9bfb-b33c122a737d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:34.934591 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1422/2826] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:26:34 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:26:34.938860 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891209e-0bc9-4772-803a-f99e69eed463 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:34.950687 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8891209e-0bc9-4772-803a-f99e69eed463 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:34.957157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fba64-be08-40e9-a81f-31c509c1ff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:34.957828 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fba64-be08-40e9-a81f-31c509c1ff8e admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:34.958131 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fba64-be08-40e9-a81f-31c509c1ff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:34.958492 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fba64-be08-40e9-a81f-31c509c1ff8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:34.959225 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fba64-be08-40e9-a81f-31c509c1ff8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:34.960927 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fba64-be08-40e9-a81f-31c509c1ff8e admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:34.972287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1405/2827] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:26:34.976868 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a67c78-def0-4409-9bfb-b33c122a737d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:34.991984 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a67c78-def0-4409-9bfb-b33c122a737d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:35.000183 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defd914-b2c0-4804-a30d-2a7e1318930c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:35.001069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defd914-b2c0-4804-a30d-2a7e1318930c admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:35.001413 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defd914-b2c0-4804-a30d-2a7e1318930c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:35.001799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defd914-b2c0-4804-a30d-2a7e1318930c admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:35.002947 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defd914-b2c0-4804-a30d-2a7e1318930c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:35.005530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defd914-b2c0-4804-a30d-2a7e1318930c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:35.021476 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1423/2828] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 11:26:35.028373 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fba64-be08-40e9-a81f-31c509c1ff8e admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:35.042665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fba64-be08-40e9-a81f-31c509c1ff8e admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:35.047929 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77b894-ea27-4cc1-a1ff-4c0e88bbed41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:35.048821 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77b894-ea27-4cc1-a1ff-4c0e88bbed41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:35.049470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77b894-ea27-4cc1-a1ff-4c0e88bbed41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:35.049759 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77b894-ea27-4cc1-a1ff-4c0e88bbed41 admin admin] PATH_INFO: `/v3/projects/fd4fc23052e44a59b11e5aef4d11b6f1/users/24a5065f381a42cc86de0fdcf25c4bb3/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:35.065496 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77b894-ea27-4cc1-a1ff-4c0e88bbed41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4fc23052e44a59b11e5aef4d11b6f1, user_id=24a5065f381a42cc86de0fdcf25c4bb3, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:35.068208 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77b894-ea27-4cc1-a1ff-4c0e88bbed41 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:35.084698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1406/2829] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:26:35 2022] PUT /identity/v3/projects/fd4fc23052e44a59b11e5aef4d11b6f1/users/24a5065f381a42cc86de0fdcf25c4bb3/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:35.096546 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97070547-8fb3-48c8-8b57-a1c73f66db0f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:35.097027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97070547-8fb3-48c8-8b57-a1c73f66db0f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:35.097508 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97070547-8fb3-48c8-8b57-a1c73f66db0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:35.125312 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-97070547-8fb3-48c8-8b57-a1c73f66db0f None None] MFA Rules not processed for user `24a5065f381a42cc86de0fdcf25c4bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:35.126568 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-97070547-8fb3-48c8-8b57-a1c73f66db0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:35.207530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1424/2830] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:35.224746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77b894-ea27-4cc1-a1ff-4c0e88bbed41 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:35.227371 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-8d77b894-ea27-4cc1-a1ff-4c0e88bbed41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:35.291165 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77b894-ea27-4cc1-a1ff-4c0e88bbed41 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:35.296086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787bfc8-713d-4530-8cf2-16c877688e94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:35.296738 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787bfc8-713d-4530-8cf2-16c877688e94 service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:35.297028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787bfc8-713d-4530-8cf2-16c877688e94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:35.297310 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787bfc8-713d-4530-8cf2-16c877688e94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:35.309157 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787bfc8-713d-4530-8cf2-16c877688e94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:35.309850 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787bfc8-713d-4530-8cf2-16c877688e94 service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:35.318939 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1407/2831] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:45.204396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:45.314807 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:58.245982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97070547-8fb3-48c8-8b57-a1c73f66db0f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:58.324849 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97070547-8fb3-48c8-8b57-a1c73f66db0f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:58.330750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:58.331794 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:58.332189 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:58.332565 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] PATH_INFO: `/v3/users/24a5065f381a42cc86de0fdcf25c4bb3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:58.334044 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a5065f381a42cc86de0fdcf25c4bb3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:58.336153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:58.391904 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a5065f381a42cc86de0fdcf25c4bb3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:58.399832 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a5065f381a42cc86de0fdcf25c4bb3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:58.400664 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a5065f381a42cc86de0fdcf25c4bb3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:58.401266 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a5065f381a42cc86de0fdcf25c4bb3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:58.414484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a5065f381a42cc86de0fdcf25c4bb3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:58.428117 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1425/2832] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:26:58 2022] DELETE /identity/v3/users/24a5065f381a42cc86de0fdcf25c4bb3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:58.432894 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9069b5-6219-4608-b72d-6f23bd41cc50 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:58.433441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9069b5-6219-4608-b72d-6f23bd41cc50 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:58.433441 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9069b5-6219-4608-b72d-6f23bd41cc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:58.451184 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0a9069b5-6219-4608-b72d-6f23bd41cc50 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:58.452385 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0a9069b5-6219-4608-b72d-6f23bd41cc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:58.520882 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1408/2833] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:58.531039 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:58.532684 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:58.596196 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7c772-c0e4-4e0b-9c7d-78bf0edab06d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:58.600234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024cb63-dd1e-4072-b28d-96a84f41ac7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:58.600879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024cb63-dd1e-4072-b28d-96a84f41ac7b service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:58.601159 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024cb63-dd1e-4072-b28d-96a84f41ac7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:58.601429 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024cb63-dd1e-4072-b28d-96a84f41ac7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:58.611563 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024cb63-dd1e-4072-b28d-96a84f41ac7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:58.613035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024cb63-dd1e-4072-b28d-96a84f41ac7b service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:58.625028 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1426/2834] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:26:59.169267 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9069b5-6219-4608-b72d-6f23bd41cc50 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:59.170921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0a9069b5-6219-4608-b72d-6f23bd41cc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:59.189012 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9069b5-6219-4608-b72d-6f23bd41cc50 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:59.194375 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:59.195103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.195386 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.195663 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] PATH_INFO: `/v3/projects/fd4fc23052e44a59b11e5aef4d11b6f1` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.197045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4fc23052e44a59b11e5aef4d11b6f1)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:59.198923 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:59.247543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4fc23052e44a59b11e5aef4d11b6f1', 'request_id': 'req-7b3ae370-e5f4-4425-8373-368f64374365'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:59.257851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4fc23052e44a59b11e5aef4d11b6f1', 'request_id': 'req-7b3ae370-e5f4-4425-8373-368f64374365'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:59.264106 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4fc23052e44a59b11e5aef4d11b6f1', 'request_id': 'req-7b3ae370-e5f4-4425-8373-368f64374365'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:59.275346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] The token cache is being invalidate because project fd4fc23052e44a59b11e5aef4d11b6f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:26:59.275632 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:59.276308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-7b3ae370-e5f4-4425-8373-368f64374365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:26:59.277955 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1409/2835] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:26:59 2022] DELETE /identity/v3/projects/fd4fc23052e44a59b11e5aef4d11b6f1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:59.283167 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81bee7-355b-4691-84f0-8ca73b7a7541 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.283694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81bee7-355b-4691-84f0-8ca73b7a7541 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.283818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa81bee7-355b-4691-84f0-8ca73b7a7541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.302216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-fa81bee7-355b-4691-84f0-8ca73b7a7541 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:59.303453 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fa81bee7-355b-4691-84f0-8ca73b7a7541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:59.320899 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:59.367649 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1427/2836] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:59.376233 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b555e3-fe6c-4970-91b2-dc32c974c3a8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.376529 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b555e3-fe6c-4970-91b2-dc32c974c3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.376811 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b555e3-fe6c-4970-91b2-dc32c974c3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.396611 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-75b555e3-fe6c-4970-91b2-dc32c974c3a8 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:59.397834 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-75b555e3-fe6c-4970-91b2-dc32c974c3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:59.406436 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1410/2837] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:59.412640 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81bee7-355b-4691-84f0-8ca73b7a7541 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:59.422736 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa81bee7-355b-4691-84f0-8ca73b7a7541 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:59.428477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e20a5-f233-4880-9b25-9683851c3768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:59.429200 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e20a5-f233-4880-9b25-9683851c3768 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.429411 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e20a5-f233-4880-9b25-9683851c3768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.429682 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e20a5-f233-4880-9b25-9683851c3768 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.431471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e20a5-f233-4880-9b25-9683851c3768 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:26:59.431471 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e20a5-f233-4880-9b25-9683851c3768 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:59.432373 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e20a5-f233-4880-9b25-9683851c3768 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:59.446857 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1428/2838] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:26:59.451350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b555e3-fe6c-4970-91b2-dc32c974c3a8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:59.465578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b555e3-fe6c-4970-91b2-dc32c974c3a8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:59.471829 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab7c39-c8cb-433e-bd8f-f1f40b67743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:59.472497 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab7c39-c8cb-433e-bd8f-f1f40b67743b admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.472781 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab7c39-c8cb-433e-bd8f-f1f40b67743b admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.473059 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab7c39-c8cb-433e-bd8f-f1f40b67743b admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.473760 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab7c39-c8cb-433e-bd8f-f1f40b67743b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:59.475726 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab7c39-c8cb-433e-bd8f-f1f40b67743b admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:59.504372 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1411/2839] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:26:59 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:26:59.508638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e20a5-f233-4880-9b25-9683851c3768 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:59.519377 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e20a5-f233-4880-9b25-9683851c3768 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:59.581145 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1de7b8-c765-4786-a6b2-c74fc83c910f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:59.581809 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1de7b8-c765-4786-a6b2-c74fc83c910f admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.582115 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1de7b8-c765-4786-a6b2-c74fc83c910f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.582398 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1de7b8-c765-4786-a6b2-c74fc83c910f admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.583050 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1de7b8-c765-4786-a6b2-c74fc83c910f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:59.584966 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1de7b8-c765-4786-a6b2-c74fc83c910f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:59.635867 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1429/2840] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:26:59 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 11:26:59.640732 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab7c39-c8cb-433e-bd8f-f1f40b67743b admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:59.653249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab7c39-c8cb-433e-bd8f-f1f40b67743b admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:59.659039 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e15b-0272-4024-bed9-497cde0398c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:59.660002 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e15b-0272-4024-bed9-497cde0398c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.660459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e15b-0272-4024-bed9-497cde0398c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.662226 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2e15b-0272-4024-bed9-497cde0398c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.663631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2e15b-0272-4024-bed9-497cde0398c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:59.668773 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2e15b-0272-4024-bed9-497cde0398c8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:59.679282 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1412/2841] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:26:59.683698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1de7b8-c765-4786-a6b2-c74fc83c910f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:59.694402 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1de7b8-c765-4786-a6b2-c74fc83c910f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:59.699289 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a615d-5088-4879-9324-06a99ba964ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:59.699940 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a615d-5088-4879-9324-06a99ba964ce admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.700216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a615d-5088-4879-9324-06a99ba964ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.700491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a615d-5088-4879-9324-06a99ba964ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.701221 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a615d-5088-4879-9324-06a99ba964ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:59.703507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a615d-5088-4879-9324-06a99ba964ce admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:59.714119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1430/2842] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:26:59.719099 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e15b-0272-4024-bed9-497cde0398c8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:59.730073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2e15b-0272-4024-bed9-497cde0398c8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:59.736116 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4113-1a7e-45aa-844c-77c3d1d73bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:59.737013 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4113-1a7e-45aa-844c-77c3d1d73bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.737292 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4113-1a7e-45aa-844c-77c3d1d73bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.737597 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4113-1a7e-45aa-844c-77c3d1d73bd8 admin admin] PATH_INFO: `/v3/projects/018bbc2d65f14431868015046e8d151c/users/551a99120e64411bbef7ac1b8161f6e9/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.754156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c4113-1a7e-45aa-844c-77c3d1d73bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=018bbc2d65f14431868015046e8d151c, user_id=551a99120e64411bbef7ac1b8161f6e9, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:59.756842 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c4113-1a7e-45aa-844c-77c3d1d73bd8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:59.772249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1413/2843] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:26:59 2022] PUT /identity/v3/projects/018bbc2d65f14431868015046e8d151c/users/551a99120e64411bbef7ac1b8161f6e9/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:26:59.782702 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9831106-68f3-4565-8604-506a15cc2be6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.782992 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9831106-68f3-4565-8604-506a15cc2be6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.783286 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9831106-68f3-4565-8604-506a15cc2be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.810968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f9831106-68f3-4565-8604-506a15cc2be6 None None] MFA Rules not processed for user `551a99120e64411bbef7ac1b8161f6e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:26:59.812763 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f9831106-68f3-4565-8604-506a15cc2be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:26:59.876261 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:59.885141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1431/2844] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:26:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:26:59.889853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4113-1a7e-45aa-844c-77c3d1d73bd8 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:26:59.913843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:26:59.959689 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4113-1a7e-45aa-844c-77c3d1d73bd8 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:26:59.965270 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e24839-40fc-4da4-b5ed-d700852ea844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:26:59.966078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e24839-40fc-4da4-b5ed-d700852ea844 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:26:59.966533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e24839-40fc-4da4-b5ed-d700852ea844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:26:59.966909 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e24839-40fc-4da4-b5ed-d700852ea844 admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:26:59.967575 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e24839-40fc-4da4-b5ed-d700852ea844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:26:59.969768 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e24839-40fc-4da4-b5ed-d700852ea844 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:26:59.998831 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1414/2845] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:26:59 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:27:00.003386 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9831106-68f3-4565-8604-506a15cc2be6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:00.063566 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9831106-68f3-4565-8604-506a15cc2be6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:00.129506 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45aedf-55cc-4179-8054-ab5e2fec35ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:00.130208 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45aedf-55cc-4179-8054-ab5e2fec35ff admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.130491 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45aedf-55cc-4179-8054-ab5e2fec35ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.130768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45aedf-55cc-4179-8054-ab5e2fec35ff admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.131428 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45aedf-55cc-4179-8054-ab5e2fec35ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:00.133356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45aedf-55cc-4179-8054-ab5e2fec35ff admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:00.156349 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1432/2846] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:27:00 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:27:00.160360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e24839-40fc-4da4-b5ed-d700852ea844 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:00.206369 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e24839-40fc-4da4-b5ed-d700852ea844 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:00.211379 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427197a8-873c-46ce-b266-66463828a3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:00.212055 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427197a8-873c-46ce-b266-66463828a3ce admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.212337 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427197a8-873c-46ce-b266-66463828a3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.212616 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427197a8-873c-46ce-b266-66463828a3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.213350 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427197a8-873c-46ce-b266-66463828a3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:00.215010 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427197a8-873c-46ce-b266-66463828a3ce admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:00.225370 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1415/2847] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:27:00.229692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45aedf-55cc-4179-8054-ab5e2fec35ff admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:00.240234 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45aedf-55cc-4179-8054-ab5e2fec35ff admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:00.245511 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb498d5-55c9-4a46-9ad8-22c0fb658668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:00.246196 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb498d5-55c9-4a46-9ad8-22c0fb658668 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.246484 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb498d5-55c9-4a46-9ad8-22c0fb658668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.246755 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb498d5-55c9-4a46-9ad8-22c0fb658668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.247500 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb498d5-55c9-4a46-9ad8-22c0fb658668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:00.249094 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb498d5-55c9-4a46-9ad8-22c0fb658668 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:00.259237 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1433/2848] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:27:00.263655 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427197a8-873c-46ce-b266-66463828a3ce admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:00.274710 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427197a8-873c-46ce-b266-66463828a3ce admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:00.279779 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb9cee-b300-4a2d-9844-b6580ddcdbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:00.280801 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb9cee-b300-4a2d-9844-b6580ddcdbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.281103 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb9cee-b300-4a2d-9844-b6580ddcdbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.281411 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb9cee-b300-4a2d-9844-b6580ddcdbe2 admin admin] PATH_INFO: `/v3/projects/6243d55139034174ac9966fe7bff5059/users/74f36507c88648cf846ea9ff91076e1c/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.298033 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb9cee-b300-4a2d-9844-b6580ddcdbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6243d55139034174ac9966fe7bff5059, user_id=74f36507c88648cf846ea9ff91076e1c, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:00.301082 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb9cee-b300-4a2d-9844-b6580ddcdbe2 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:00.318873 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1416/2849] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:27:00 2022] PUT /identity/v3/projects/6243d55139034174ac9966fe7bff5059/users/74f36507c88648cf846ea9ff91076e1c/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:27:00.330490 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a3c6a-4d27-4331-a796-867ed691b21a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.331019 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a3c6a-4d27-4331-a796-867ed691b21a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.331327 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a3c6a-4d27-4331-a796-867ed691b21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.364227 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c41a3c6a-4d27-4331-a796-867ed691b21a None None] MFA Rules not processed for user `74f36507c88648cf846ea9ff91076e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:27:00.365485 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c41a3c6a-4d27-4331-a796-867ed691b21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:00.451901 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1434/2850] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:27:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:27:00.457335 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb9cee-b300-4a2d-9844-b6580ddcdbe2 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:00.523089 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb9cee-b300-4a2d-9844-b6580ddcdbe2 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:00.529429 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf1f3c-a03b-4c5e-896f-5fc77f71d38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:00.530115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf1f3c-a03b-4c5e-896f-5fc77f71d38d admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.530433 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf1f3c-a03b-4c5e-896f-5fc77f71d38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.530986 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf1f3c-a03b-4c5e-896f-5fc77f71d38d admin admin] PATH_INFO: `/v3/projects` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.531461 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf1f3c-a03b-4c5e-896f-5fc77f71d38d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:00.533403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf1f3c-a03b-4c5e-896f-5fc77f71d38d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:00.563544 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1417/2851] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:27:00 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:27:00.568128 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a3c6a-4d27-4331-a796-867ed691b21a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:00.580694 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a3c6a-4d27-4331-a796-867ed691b21a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:00.610686 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:00.648884 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6cade-8c11-4abb-b66e-4af57f3c729a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:00.649560 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6cade-8c11-4abb-b66e-4af57f3c729a admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.649838 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6cade-8c11-4abb-b66e-4af57f3c729a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.650113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6cade-8c11-4abb-b66e-4af57f3c729a admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.650818 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6cade-8c11-4abb-b66e-4af57f3c729a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:00.652732 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6cade-8c11-4abb-b66e-4af57f3c729a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:00.674891 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1435/2852] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:27:00 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:27:00.680560 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf1f3c-a03b-4c5e-896f-5fc77f71d38d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:00.694450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf1f3c-a03b-4c5e-896f-5fc77f71d38d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:00.700346 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef6602-957a-4c70-a91d-8c2ef92336bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:00.701020 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef6602-957a-4c70-a91d-8c2ef92336bd admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.701305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef6602-957a-4c70-a91d-8c2ef92336bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.701619 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef6602-957a-4c70-a91d-8c2ef92336bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.702390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef6602-957a-4c70-a91d-8c2ef92336bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:00.704014 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef6602-957a-4c70-a91d-8c2ef92336bd admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:00.716678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1418/2853] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:27:00.721874 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6cade-8c11-4abb-b66e-4af57f3c729a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:00.737730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6cade-8c11-4abb-b66e-4af57f3c729a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:00.745599 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73808bbb-8596-4b3f-a4c4-1a67edd23c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:00.746610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73808bbb-8596-4b3f-a4c4-1a67edd23c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.746892 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73808bbb-8596-4b3f-a4c4-1a67edd23c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.747166 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73808bbb-8596-4b3f-a4c4-1a67edd23c73 admin admin] PATH_INFO: `/v3/projects/1a58f14147be43d1b5ad7dfbb824aff6/users/96847a5ec7964c63a5b494355a3253f3/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.764798 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73808bbb-8596-4b3f-a4c4-1a67edd23c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a58f14147be43d1b5ad7dfbb824aff6, user_id=96847a5ec7964c63a5b494355a3253f3, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:00.768285 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73808bbb-8596-4b3f-a4c4-1a67edd23c73 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:00.783401 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1436/2854] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:27:00 2022] PUT /identity/v3/projects/1a58f14147be43d1b5ad7dfbb824aff6/users/96847a5ec7964c63a5b494355a3253f3/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:27:00.790255 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef6602-957a-4c70-a91d-8c2ef92336bd admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:00.807670 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef6602-957a-4c70-a91d-8c2ef92336bd admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:00.818340 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:00.869317 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03460a-9aa9-436a-a1ac-6f4fe213a473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:00.869985 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03460a-9aa9-436a-a1ac-6f4fe213a473 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.870296 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03460a-9aa9-436a-a1ac-6f4fe213a473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.870573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03460a-9aa9-436a-a1ac-6f4fe213a473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.871305 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03460a-9aa9-436a-a1ac-6f4fe213a473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:00.872958 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03460a-9aa9-436a-a1ac-6f4fe213a473 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:00.883490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1419/2855] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:27:00.888300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73808bbb-8596-4b3f-a4c4-1a67edd23c73 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:00.899228 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73808bbb-8596-4b3f-a4c4-1a67edd23c73 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:00.904564 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f47690-85c7-403c-bae1-82e99fec0ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:00.905459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47690-85c7-403c-bae1-82e99fec0ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.905742 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47690-85c7-403c-bae1-82e99fec0ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.906153 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47690-85c7-403c-bae1-82e99fec0ec3 admin admin] PATH_INFO: `/v3/projects/1a58f14147be43d1b5ad7dfbb824aff6/users/96847a5ec7964c63a5b494355a3253f3/roles/030ee64221f74a029d408ae3bdb824a1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.908532 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f47690-85c7-403c-bae1-82e99fec0ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a58f14147be43d1b5ad7dfbb824aff6, user_id=96847a5ec7964c63a5b494355a3253f3, role_id=030ee64221f74a029d408ae3bdb824a1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:00.911222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f47690-85c7-403c-bae1-82e99fec0ec3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:00.922768 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1437/2856] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:27:00 2022] PUT /identity/v3/projects/1a58f14147be43d1b5ad7dfbb824aff6/users/96847a5ec7964c63a5b494355a3253f3/roles/030ee64221f74a029d408ae3bdb824a1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:27:00.933454 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a45b25-9a29-455c-8dbe-dc10354a5011 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:00.933718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a45b25-9a29-455c-8dbe-dc10354a5011 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:00.934080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a45b25-9a29-455c-8dbe-dc10354a5011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:00.963090 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-b2a45b25-9a29-455c-8dbe-dc10354a5011 None None] MFA Rules not processed for user `96847a5ec7964c63a5b494355a3253f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:27:00.964304 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b2a45b25-9a29-455c-8dbe-dc10354a5011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:01.040724 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1420/2857] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:27:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:27:01.054948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f47690-85c7-403c-bae1-82e99fec0ec3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:01.057584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-09f47690-85c7-403c-bae1-82e99fec0ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:01.127519 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f47690-85c7-403c-bae1-82e99fec0ec3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:01.131741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5ea26-191e-4f61-ab5b-2e9eef3320f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:01.132432 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5ea26-191e-4f61-ab5b-2e9eef3320f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:01.132774 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5ea26-191e-4f61-ab5b-2e9eef3320f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:01.133113 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5ea26-191e-4f61-ab5b-2e9eef3320f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:01.207192 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5ea26-191e-4f61-ab5b-2e9eef3320f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:01.208655 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5ea26-191e-4f61-ab5b-2e9eef3320f9 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:01.218217 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1438/2858] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:27:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:27:02.754419 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a45b25-9a29-455c-8dbe-dc10354a5011 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:02.756401 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-b2a45b25-9a29-455c-8dbe-dc10354a5011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:02.772113 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a45b25-9a29-455c-8dbe-dc10354a5011 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:02.776145 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b6e56-e7dc-456e-8c67-aa0ab0d0561f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:02.776797 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b6e56-e7dc-456e-8c67-aa0ab0d0561f service cinder] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:02.777078 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b6e56-e7dc-456e-8c67-aa0ab0d0561f service cinder] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:02.777348 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b6e56-e7dc-456e-8c67-aa0ab0d0561f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:02.834288 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b6e56-e7dc-456e-8c67-aa0ab0d0561f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:02.835694 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b6e56-e7dc-456e-8c67-aa0ab0d0561f service cinder] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:02.845138 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1421/2859] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:27:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:27:03.425481 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5ea26-191e-4f61-ab5b-2e9eef3320f9 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:03.435333 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5ea26-191e-4f61-ab5b-2e9eef3320f9 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:03.439975 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e3c68-2781-4e32-8427-9ed081819c3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:03.440608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e3c68-2781-4e32-8427-9ed081819c3e service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:03.440885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e3c68-2781-4e32-8427-9ed081819c3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:03.441342 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e3c68-2781-4e32-8427-9ed081819c3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:03.456177 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e3c68-2781-4e32-8427-9ed081819c3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:03.457812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e3c68-2781-4e32-8427-9ed081819c3e service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:03.467754 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1439/2860] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:27:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:27:07.416144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b6e56-e7dc-456e-8c67-aa0ab0d0561f service cinder] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:07.453631 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=100 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:07.485833 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b6e56-e7dc-456e-8c67-aa0ab0d0561f service cinder] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:07.490905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:07.491798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:07.492083 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:07.492343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] PATH_INFO: `/v3/users/551a99120e64411bbef7ac1b8161f6e9` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:07.493698 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=551a99120e64411bbef7ac1b8161f6e9)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:07.495728 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:07.545519 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '551a99120e64411bbef7ac1b8161f6e9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:07.546214 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '551a99120e64411bbef7ac1b8161f6e9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:07.556032 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '551a99120e64411bbef7ac1b8161f6e9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:07.557028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '551a99120e64411bbef7ac1b8161f6e9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:07.565578 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '551a99120e64411bbef7ac1b8161f6e9'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:07.583490 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1422/2861] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:27:07 2022] DELETE /identity/v3/users/551a99120e64411bbef7ac1b8161f6e9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:27:07.588335 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10953afe-7579-4911-b614-65038a75f095 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:07.588830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10953afe-7579-4911-b614-65038a75f095 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:07.588934 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10953afe-7579-4911-b614-65038a75f095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:07.607446 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-10953afe-7579-4911-b614-65038a75f095 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:27:07.608637 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-10953afe-7579-4911-b614-65038a75f095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:07.622970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:07.679524 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1440/2862] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:27:07.692334 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:07.694037 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:07.806514 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce2ab0-0c12-4cdf-9834-5399ca261dcb admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:07.811542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789308cc-c2ce-483f-9265-9a0eb65298bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:07.812194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789308cc-c2ce-483f-9265-9a0eb65298bb service neutron] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:07.812470 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789308cc-c2ce-483f-9265-9a0eb65298bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:07.812740 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789308cc-c2ce-483f-9265-9a0eb65298bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:07.822374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789308cc-c2ce-483f-9265-9a0eb65298bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:07.823823 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789308cc-c2ce-483f-9265-9a0eb65298bb service neutron] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:07.833445 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1423/2863] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:27:08.118700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10953afe-7579-4911-b614-65038a75f095 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:08.120168 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-10953afe-7579-4911-b614-65038a75f095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:08.138029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10953afe-7579-4911-b614-65038a75f095 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:08.145789 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:08.147299 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:08.147735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:08.148063 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] PATH_INFO: `/v3/projects/018bbc2d65f14431868015046e8d151c` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:08.149644 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=018bbc2d65f14431868015046e8d151c)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:08.151617 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:08.201081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '018bbc2d65f14431868015046e8d151c', 'request_id': 'req-717c9e20-f501-4a7a-a0e7-9871974b1d3e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.208797 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '018bbc2d65f14431868015046e8d151c', 'request_id': 'req-717c9e20-f501-4a7a-a0e7-9871974b1d3e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.214530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '018bbc2d65f14431868015046e8d151c', 'request_id': 'req-717c9e20-f501-4a7a-a0e7-9871974b1d3e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.226078 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] The token cache is being invalidate because project 018bbc2d65f14431868015046e8d151c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:27:08.226383 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.227090 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.228700 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1441/2864] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:27:08 2022] DELETE /identity/v3/projects/018bbc2d65f14431868015046e8d151c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:27:08.232887 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789308cc-c2ce-483f-9265-9a0eb65298bb service neutron] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:08.243115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789308cc-c2ce-483f-9265-9a0eb65298bb service neutron] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:08.256252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:08.305119 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:08.306031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:08.306343 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:08.306612 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] PATH_INFO: `/v3/users/74f36507c88648cf846ea9ff91076e1c` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:08.307859 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74f36507c88648cf846ea9ff91076e1c)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:08.310144 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:08.368600 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74f36507c88648cf846ea9ff91076e1c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.369252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74f36507c88648cf846ea9ff91076e1c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.377684 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74f36507c88648cf846ea9ff91076e1c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.378359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74f36507c88648cf846ea9ff91076e1c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.385843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74f36507c88648cf846ea9ff91076e1c'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.403504 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1424/2865] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:27:08 2022] DELETE /identity/v3/users/74f36507c88648cf846ea9ff91076e1c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:27:08.693956 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:08.695548 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:08.710030 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c9e20-f501-4a7a-a0e7-9871974b1d3e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:08.769817 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:08.770671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:08.770960 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:08.771241 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] PATH_INFO: `/v3/projects/6243d55139034174ac9966fe7bff5059` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:08.772457 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6243d55139034174ac9966fe7bff5059)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:08.774438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:08.820460 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:08.824303 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6243d55139034174ac9966fe7bff5059', 'request_id': 'req-e3f56d8a-1cbb-4fe8-916a-acddf521b327'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.833351 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6243d55139034174ac9966fe7bff5059', 'request_id': 'req-e3f56d8a-1cbb-4fe8-916a-acddf521b327'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.839477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6243d55139034174ac9966fe7bff5059', 'request_id': 'req-e3f56d8a-1cbb-4fe8-916a-acddf521b327'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.851946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] The token cache is being invalidate because project 6243d55139034174ac9966fe7bff5059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:27:08.852226 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.852885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.854537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1442/2866] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:27:08 2022] DELETE /identity/v3/projects/6243d55139034174ac9966fe7bff5059 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:27:08.858595 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:08.860081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:08.873149 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c0dc1-91a0-40bd-81d1-96bbd2e4ced5 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:08.932979 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:08.933862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:08.934164 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:08.934437 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] PATH_INFO: `/v3/users/96847a5ec7964c63a5b494355a3253f3` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:08.935713 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=96847a5ec7964c63a5b494355a3253f3)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:08.937668 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:08.984395 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:08.992477 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96847a5ec7964c63a5b494355a3253f3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:08.993447 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96847a5ec7964c63a5b494355a3253f3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:09.002899 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96847a5ec7964c63a5b494355a3253f3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:09.003566 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96847a5ec7964c63a5b494355a3253f3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:09.011062 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-a3dd94e5-1a97-45f6-b673-3781ef6f28bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96847a5ec7964c63a5b494355a3253f3'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:09.029898 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1425/2867] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:27:08 2022] DELETE /identity/v3/users/96847a5ec7964c63a5b494355a3253f3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:27:09.333193 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:09.334848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:09.349074 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f56d8a-1cbb-4fe8-916a-acddf521b327 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:09.409077 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:09.410015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:09.410259 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:09.410567 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] PATH_INFO: `/v3/projects/1a58f14147be43d1b5ad7dfbb824aff6` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:09.411808 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a58f14147be43d1b5ad7dfbb824aff6)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:09.413753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:09.468280 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a58f14147be43d1b5ad7dfbb824aff6', 'request_id': 'req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:09.477473 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a58f14147be43d1b5ad7dfbb824aff6', 'request_id': 'req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:09.483210 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a58f14147be43d1b5ad7dfbb824aff6', 'request_id': 'req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:09.495051 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] The token cache is being invalidate because project 1a58f14147be43d1b5ad7dfbb824aff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:27:09.495293 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:09.496096 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67e1d92e-632d-4ff3-b279-ce73c99aa6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:27:09.497800 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1443/2868] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:27:09 2022] DELETE /identity/v3/projects/1a58f14147be43d1b5ad7dfbb824aff6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 11:27:09.502998 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c499c-b69d-44d4-a34e-845c04b28ce7 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:09.503308 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c499c-b69d-44d4-a34e-845c04b28ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:09.503608 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c499c-b69d-44d4-a34e-845c04b28ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:09.523869 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-831c499c-b69d-44d4-a34e-845c04b28ce7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:27:09.525081 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-831c499c-b69d-44d4-a34e-845c04b28ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:09.596889 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1426/2869] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:27:09.606525 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f472f7-979a-493e-b54f-883e5f1fca68 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:09.606825 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f472f7-979a-493e-b54f-883e5f1fca68 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:09.607085 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f472f7-979a-493e-b54f-883e5f1fca68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:09.626438 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-87f472f7-979a-493e-b54f-883e5f1fca68 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:27:09.627671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-87f472f7-979a-493e-b54f-883e5f1fca68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:09.637240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1444/2870] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:27:09.643065 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c499c-b69d-44d4-a34e-845c04b28ce7 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:09.657260 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c499c-b69d-44d4-a34e-845c04b28ce7 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:09.665746 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac9f8b-73d0-4cba-90e9-804b52e67d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:09.666473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac9f8b-73d0-4cba-90e9-804b52e67d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:09.666757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac9f8b-73d0-4cba-90e9-804b52e67d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:09.667031 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac9f8b-73d0-4cba-90e9-804b52e67d5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:09.667798 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac9f8b-73d0-4cba-90e9-804b52e67d5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:27:09.668111 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac9f8b-73d0-4cba-90e9-804b52e67d5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:09.669756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac9f8b-73d0-4cba-90e9-804b52e67d5d admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:09.682191 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1427/2871] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:27:09.686613 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f472f7-979a-493e-b54f-883e5f1fca68 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:09.697887 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f472f7-979a-493e-b54f-883e5f1fca68 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:09.703220 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3f41a-8b67-46cf-a069-0ba601dd2f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:09.703893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3f41a-8b67-46cf-a069-0ba601dd2f8c admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:09.704178 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3f41a-8b67-46cf-a069-0ba601dd2f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:09.704463 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3f41a-8b67-46cf-a069-0ba601dd2f8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:09.705155 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3f41a-8b67-46cf-a069-0ba601dd2f8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:09.707127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3f41a-8b67-46cf-a069-0ba601dd2f8c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:09.735787 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1445/2872] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:27:09 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:27:09.740607 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac9f8b-73d0-4cba-90e9-804b52e67d5d admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:09.750718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac9f8b-73d0-4cba-90e9-804b52e67d5d admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:09.814984 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3ea58-4c2b-4742-a0ce-b164fe9d6577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:09.815843 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3ea58-4c2b-4742-a0ce-b164fe9d6577 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:09.816206 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3ea58-4c2b-4742-a0ce-b164fe9d6577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:09.816524 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3ea58-4c2b-4742-a0ce-b164fe9d6577 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:09.817579 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3ea58-4c2b-4742-a0ce-b164fe9d6577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:09.819747 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3ea58-4c2b-4742-a0ce-b164fe9d6577 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:09.846822 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1428/2873] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:27:09 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:27:09.852107 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3f41a-8b67-46cf-a069-0ba601dd2f8c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:09.868741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3f41a-8b67-46cf-a069-0ba601dd2f8c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:09.877717 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f427f70-5005-47ba-9271-d7c3aea69fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:09.878735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427f70-5005-47ba-9271-d7c3aea69fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:09.879097 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427f70-5005-47ba-9271-d7c3aea69fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:09.879448 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427f70-5005-47ba-9271-d7c3aea69fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:09.880959 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f427f70-5005-47ba-9271-d7c3aea69fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:09.882610 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f427f70-5005-47ba-9271-d7c3aea69fa1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:09.893397 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1446/2874] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:27:09.898408 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3ea58-4c2b-4742-a0ce-b164fe9d6577 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:09.909331 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3ea58-4c2b-4742-a0ce-b164fe9d6577 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:09.915299 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610d3e1-e1cb-448c-ab53-2ca320957df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:09.915982 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d3e1-e1cb-448c-ab53-2ca320957df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:09.916283 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d3e1-e1cb-448c-ab53-2ca320957df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:09.916565 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610d3e1-e1cb-448c-ab53-2ca320957df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:09.917303 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d3e1-e1cb-448c-ab53-2ca320957df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:09.918962 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610d3e1-e1cb-448c-ab53-2ca320957df8 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:09.930573 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1429/2875] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:27:09.935606 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f427f70-5005-47ba-9271-d7c3aea69fa1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:09.951590 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f427f70-5005-47ba-9271-d7c3aea69fa1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:09.959661 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b76fb8-397a-4991-bf17-5521688e0793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:09.960556 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b76fb8-397a-4991-bf17-5521688e0793 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:09.960844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b76fb8-397a-4991-bf17-5521688e0793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:09.961118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b76fb8-397a-4991-bf17-5521688e0793 admin admin] PATH_INFO: `/v3/projects/d87dd26331ac46338a1d19800fd1c067/users/bf758ca5ff364fa6b56e841128a1b4fb/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:09.977714 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b76fb8-397a-4991-bf17-5521688e0793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87dd26331ac46338a1d19800fd1c067, user_id=bf758ca5ff364fa6b56e841128a1b4fb, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:09.980447 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b76fb8-397a-4991-bf17-5521688e0793 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:09.993520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:10.001721 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1447/2876] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:27:09 2022] PUT /identity/v3/projects/d87dd26331ac46338a1d19800fd1c067/users/bf758ca5ff364fa6b56e841128a1b4fb/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:27:10.012399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce889d3c-18e0-41de-93ac-0430cf7d3965 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:10.012868 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce889d3c-18e0-41de-93ac-0430cf7d3965 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:10.013328 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce889d3c-18e0-41de-93ac-0430cf7d3965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:10.035795 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ce889d3c-18e0-41de-93ac-0430cf7d3965 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:27:10.037006 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ce889d3c-18e0-41de-93ac-0430cf7d3965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:10.056360 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:10.104182 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1430/2877] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:27:10.115699 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b76fb8-397a-4991-bf17-5521688e0793 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:10.117726 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d9b76fb8-397a-4991-bf17-5521688e0793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:10.184598 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b76fb8-397a-4991-bf17-5521688e0793 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:10.189035 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387e7e4-6e40-49f0-9f27-456c08d34375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:10.189688 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387e7e4-6e40-49f0-9f27-456c08d34375 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:10.189973 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387e7e4-6e40-49f0-9f27-456c08d34375 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:10.190283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387e7e4-6e40-49f0-9f27-456c08d34375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:10.200666 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387e7e4-6e40-49f0-9f27-456c08d34375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:10.202119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387e7e4-6e40-49f0-9f27-456c08d34375 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:10.212939 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1448/2878] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:27:10.887249 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b564d31-54de-4433-8f18-dd1d2df6e6c9 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:10.887853 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b564d31-54de-4433-8f18-dd1d2df6e6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:10.888459 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b564d31-54de-4433-8f18-dd1d2df6e6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:10.919865 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-0b564d31-54de-4433-8f18-dd1d2df6e6c9 None None] MFA Rules not processed for user `bf758ca5ff364fa6b56e841128a1b4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:27:10.921073 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-0b564d31-54de-4433-8f18-dd1d2df6e6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:11.002333 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1431/2879] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:27:11.013248 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387e7e4-6e40-49f0-9f27-456c08d34375 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:11.014859 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b387e7e4-6e40-49f0-9f27-456c08d34375 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:11.075223 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387e7e4-6e40-49f0-9f27-456c08d34375 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:11.080356 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823801b0-f727-4a44-b84f-c8cb9b6c5f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:11.081014 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823801b0-f727-4a44-b84f-c8cb9b6c5f05 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:11.081300 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823801b0-f727-4a44-b84f-c8cb9b6c5f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:11.081569 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823801b0-f727-4a44-b84f-c8cb9b6c5f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:11.092257 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823801b0-f727-4a44-b84f-c8cb9b6c5f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:11.093699 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823801b0-f727-4a44-b84f-c8cb9b6c5f05 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:11.103843 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1449/2880] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:27:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:27:12.489389 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0514b4d-fb61-42fc-8c12-8332725e1104 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:12.489971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0514b4d-fb61-42fc-8c12-8332725e1104 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:12.489971 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0514b4d-fb61-42fc-8c12-8332725e1104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:12.508921 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-e0514b4d-fb61-42fc-8c12-8332725e1104 None None] MFA Rules not processed for user `bf758ca5ff364fa6b56e841128a1b4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:27:12.510252 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e0514b4d-fb61-42fc-8c12-8332725e1104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:12.518403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1432/2881] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:27:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:27:14.886697 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823801b0-f727-4a44-b84f-c8cb9b6c5f05 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:14.888283 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-823801b0-f727-4a44-b84f-c8cb9b6c5f05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:14.902346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823801b0-f727-4a44-b84f-c8cb9b6c5f05 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:14.907103 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39de06e-e5e4-4494-87dc-fb5aadfacf4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:14.907982 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39de06e-e5e4-4494-87dc-fb5aadfacf4b service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:14.908141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39de06e-e5e4-4494-87dc-fb5aadfacf4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:14.908406 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39de06e-e5e4-4494-87dc-fb5aadfacf4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:14.911426 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d39de06e-e5e4-4494-87dc-fb5aadfacf4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:14.922848 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39de06e-e5e4-4494-87dc-fb5aadfacf4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:14.924659 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39de06e-e5e4-4494-87dc-fb5aadfacf4b service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:14.933172 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1450/2882] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:27:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:27:15.605533 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0514b4d-fb61-42fc-8c12-8332725e1104 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:15.607163 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-e0514b4d-fb61-42fc-8c12-8332725e1104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:15.689052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0514b4d-fb61-42fc-8c12-8332725e1104 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:15.697439 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072964c-e59a-41d9-bb14-04725c550486 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:15.698374 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072964c-e59a-41d9-bb14-04725c550486 service nova] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:15.698652 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072964c-e59a-41d9-bb14-04725c550486 service nova] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:15.698934 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072964c-e59a-41d9-bb14-04725c550486 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:15.700884 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c072964c-e59a-41d9-bb14-04725c550486 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:15.713975 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072964c-e59a-41d9-bb14-04725c550486 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:15.715648 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072964c-e59a-41d9-bb14-04725c550486 service nova] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:15.728443 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1433/2883] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:27:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:27:21.180058 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39de06e-e5e4-4494-87dc-fb5aadfacf4b service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:21.181970 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d39de06e-e5e4-4494-87dc-fb5aadfacf4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:21.200708 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39de06e-e5e4-4494-87dc-fb5aadfacf4b service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:21.207594 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbf3cd-3f8a-4d58-8bd9-7c9f19400fe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37b1f13aa474493d83b3a16f76949fbc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:21.208321 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbf3cd-3f8a-4d58-8bd9-7c9f19400fe0 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:21.208595 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbf3cd-3f8a-4d58-8bd9-7c9f19400fe0 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:21.208844 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbf3cd-3f8a-4d58-8bd9-7c9f19400fe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:21.211060 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-60cbf3cd-3f8a-4d58-8bd9-7c9f19400fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:21.224586 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbf3cd-3f8a-4d58-8bd9-7c9f19400fe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:21.226192 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cbf3cd-3f8a-4d58-8bd9-7c9f19400fe0 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:21.237906 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1451/2884] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:27:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:27:25.722324 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=86 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:26.809045 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a05e2-8cb5-48c8-8b96-aa9a8fd6eb4b None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:26.809733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a05e2-8cb5-48c8-8b96-aa9a8fd6eb4b None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:26.809733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a05e2-8cb5-48c8-8b96-aa9a8fd6eb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:26.828816 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-ae1a05e2-8cb5-48c8-8b96-aa9a8fd6eb4b None None] MFA Rules not processed for user `bf758ca5ff364fa6b56e841128a1b4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:27:26.830063 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-ae1a05e2-8cb5-48c8-8b96-aa9a8fd6eb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:26.838194 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1434/2885] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:27:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:27:31.233262 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=64 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:36.823028 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:42.389029 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbf3cd-3f8a-4d58-8bd9-7c9f19400fe0 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:42.390630 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-60cbf3cd-3f8a-4d58-8bd9-7c9f19400fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:42.486318 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbf3cd-3f8a-4d58-8bd9-7c9f19400fe0 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:42.496535 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056d513-08ae-4b70-b342-f3539e93d785 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:42.497414 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056d513-08ae-4b70-b342-f3539e93d785 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:42.497816 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056d513-08ae-4b70-b342-f3539e93d785 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:42.498306 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056d513-08ae-4b70-b342-f3539e93d785 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:42.501584 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1056d513-08ae-4b70-b342-f3539e93d785 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:42.517076 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056d513-08ae-4b70-b342-f3539e93d785 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:42.518725 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056d513-08ae-4b70-b342-f3539e93d785 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:42.529370 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1452/2886] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:27:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:27:49.327086 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803259e7-c4fe-4c65-b5cb-0641690e50de None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:49.327718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803259e7-c4fe-4c65-b5cb-0641690e50de None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:49.327718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803259e7-c4fe-4c65-b5cb-0641690e50de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:49.390722 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-803259e7-c4fe-4c65-b5cb-0641690e50de None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:27:49.391948 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-803259e7-c4fe-4c65-b5cb-0641690e50de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:49.469115 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1435/2887] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:27:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:27:49.519968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056d513-08ae-4b70-b342-f3539e93d785 service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:27:49.521661 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1056d513-08ae-4b70-b342-f3539e93d785 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:27:49.619330 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056d513-08ae-4b70-b342-f3539e93d785 service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:27:49.625914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5263670-7a79-435b-81fe-7b9cb4e92dab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:27:49.626608 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5263670-7a79-435b-81fe-7b9cb4e92dab service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:27:49.626893 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5263670-7a79-435b-81fe-7b9cb4e92dab service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:27:49.627172 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5263670-7a79-435b-81fe-7b9cb4e92dab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:27:49.636396 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5263670-7a79-435b-81fe-7b9cb4e92dab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:27:49.638112 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5263670-7a79-435b-81fe-7b9cb4e92dab service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:27:49.646638 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1453/2888] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:27:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:27:59.465098 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=44 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:27:59.643385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=82 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:28:04.067972 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e172d-1c02-4d14-9b5e-f35830ec0019 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:04.068465 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e172d-1c02-4d14-9b5e-f35830ec0019 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:04.068586 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e172d-1c02-4d14-9b5e-f35830ec0019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:04.088262 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-c47e172d-1c02-4d14-9b5e-f35830ec0019 None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:28:04.089505 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-c47e172d-1c02-4d14-9b5e-f35830ec0019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:04.102622 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1436/2889] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jun 20 11:28:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:28:04.115369 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5263670-7a79-435b-81fe-7b9cb4e92dab service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:04.127137 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5263670-7a79-435b-81fe-7b9cb4e92dab service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:04.132537 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26980d-c34f-4109-aafa-8e8d0b7edbe7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:04.133191 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26980d-c34f-4109-aafa-8e8d0b7edbe7 service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:04.133477 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26980d-c34f-4109-aafa-8e8d0b7edbe7 service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:04.133753 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26980d-c34f-4109-aafa-8e8d0b7edbe7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:04.143069 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26980d-c34f-4109-aafa-8e8d0b7edbe7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:04.144658 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26980d-c34f-4109-aafa-8e8d0b7edbe7 service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:04.153639 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1454/2890] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:28:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:28:06.155568 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e172d-1c02-4d14-9b5e-f35830ec0019 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:06.167399 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e172d-1c02-4d14-9b5e-f35830ec0019 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:06.172835 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:06.173765 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:06.174047 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:06.174355 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] PATH_INFO: `/v3/users/bf758ca5ff364fa6b56e841128a1b4fb` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:06.175542 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf758ca5ff364fa6b56e841128a1b4fb)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:06.177491 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:06.228321 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf758ca5ff364fa6b56e841128a1b4fb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:06.229048 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf758ca5ff364fa6b56e841128a1b4fb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:06.236940 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf758ca5ff364fa6b56e841128a1b4fb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:06.237756 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf758ca5ff364fa6b56e841128a1b4fb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:06.245592 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-1df1a519-acdc-4f85-8eac-726e008bed49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf758ca5ff364fa6b56e841128a1b4fb'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:06.264650 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1437/2891] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:28:06 2022] DELETE /identity/v3/users/bf758ca5ff364fa6b56e841128a1b4fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:28:06.462081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26980d-c34f-4109-aafa-8e8d0b7edbe7 service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:06.479830 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26980d-c34f-4109-aafa-8e8d0b7edbe7 service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:06.545260 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:06.546240 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:06.546520 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:06.546799 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] PATH_INFO: `/v3/projects/d87dd26331ac46338a1d19800fd1c067` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:06.548365 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d87dd26331ac46338a1d19800fd1c067)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:06.550332 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:06.598757 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd87dd26331ac46338a1d19800fd1c067', 'request_id': 'req-fc20509c-1c92-49d5-9800-766abfbdb367'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:06.610547 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd87dd26331ac46338a1d19800fd1c067', 'request_id': 'req-fc20509c-1c92-49d5-9800-766abfbdb367'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:06.617631 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd87dd26331ac46338a1d19800fd1c067', 'request_id': 'req-fc20509c-1c92-49d5-9800-766abfbdb367'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:06.630385 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] The token cache is being invalidate because project d87dd26331ac46338a1d19800fd1c067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:28:06.630603 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:06.631316 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc20509c-1c92-49d5-9800-766abfbdb367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:06.633004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1455/2892] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:28:06 2022] DELETE /identity/v3/projects/d87dd26331ac46338a1d19800fd1c067 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:28:06.638872 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f2db9-03d2-4a37-81c7-076b6ccfbcb8 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:06.639156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f2db9-03d2-4a37-81c7-076b6ccfbcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:06.639432 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f2db9-03d2-4a37-81c7-076b6ccfbcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:06.659532 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-030f2db9-03d2-4a37-81c7-076b6ccfbcb8 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:28:06.660757 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-030f2db9-03d2-4a37-81c7-076b6ccfbcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:06.727855 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1438/2893] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:28:06.739467 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b53eb-7cfd-44cc-8faa-33bc53e4d2f7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:06.739851 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b53eb-7cfd-44cc-8faa-33bc53e4d2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:06.740324 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b53eb-7cfd-44cc-8faa-33bc53e4d2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:06.761449 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d03b53eb-7cfd-44cc-8faa-33bc53e4d2f7 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:28:06.762727 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d03b53eb-7cfd-44cc-8faa-33bc53e4d2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:06.772222 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1456/2894] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:28:06.777621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f2db9-03d2-4a37-81c7-076b6ccfbcb8 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:06.799483 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f2db9-03d2-4a37-81c7-076b6ccfbcb8 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:06.805905 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568c1da-1dad-4abe-b9c4-1cf421852d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:06.806606 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568c1da-1dad-4abe-b9c4-1cf421852d28 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:06.806883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568c1da-1dad-4abe-b9c4-1cf421852d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:06.807148 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568c1da-1dad-4abe-b9c4-1cf421852d28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:06.807883 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568c1da-1dad-4abe-b9c4-1cf421852d28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80132) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 11:28:06.808209 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568c1da-1dad-4abe-b9c4-1cf421852d28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:06.810438 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568c1da-1dad-4abe-b9c4-1cf421852d28 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:06.823368 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1439/2895] 10.222.0.33 () {58 vars in 1152 bytes} [Mon Jun 20 11:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 11:28:06.828127 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b53eb-7cfd-44cc-8faa-33bc53e4d2f7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:06.838407 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b53eb-7cfd-44cc-8faa-33bc53e4d2f7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:06.843750 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490c99-f0a3-4405-b7d6-61823d15ef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:06.844421 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490c99-f0a3-4405-b7d6-61823d15ef05 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:06.844703 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490c99-f0a3-4405-b7d6-61823d15ef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:06.844986 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490c99-f0a3-4405-b7d6-61823d15ef05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:06.845671 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d490c99-f0a3-4405-b7d6-61823d15ef05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:06.847620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d490c99-f0a3-4405-b7d6-61823d15ef05 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:06.876459 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1457/2896] 10.222.0.33 () {60 vars in 1152 bytes} [Mon Jun 20 11:28:06 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:28:06.881156 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568c1da-1dad-4abe-b9c4-1cf421852d28 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:06.891681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568c1da-1dad-4abe-b9c4-1cf421852d28 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:06.941035 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:28:06.951681 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea50722-0b8e-41d4-a998-813e2a5de262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:06.952359 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea50722-0b8e-41d4-a998-813e2a5de262 admin admin] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:06.952644 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea50722-0b8e-41d4-a998-813e2a5de262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:06.952924 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea50722-0b8e-41d4-a998-813e2a5de262 admin admin] PATH_INFO: `/v3/users` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:06.953621 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea50722-0b8e-41d4-a998-813e2a5de262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:06.955662 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea50722-0b8e-41d4-a998-813e2a5de262 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:06.976959 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1440/2897] 10.222.0.33 () {60 vars in 1143 bytes} [Mon Jun 20 11:28:06 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 11:28:06.981395 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490c99-f0a3-4405-b7d6-61823d15ef05 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:06.991801 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490c99-f0a3-4405-b7d6-61823d15ef05 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:06.997187 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435869bb-1bec-41df-aa14-915ab5a3fec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:06.997856 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435869bb-1bec-41df-aa14-915ab5a3fec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:06.998141 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435869bb-1bec-41df-aa14-915ab5a3fec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:06.998427 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435869bb-1bec-41df-aa14-915ab5a3fec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:06.999150 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435869bb-1bec-41df-aa14-915ab5a3fec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:07.000756 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435869bb-1bec-41df-aa14-915ab5a3fec4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:07.012871 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1458/2898] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:28:07.017056 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea50722-0b8e-41d4-a998-813e2a5de262 admin admin] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:07.027720 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea50722-0b8e-41d4-a998-813e2a5de262 admin admin] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:07.033050 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb5f05-f3e0-4c8b-abb9-e95c40921aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:07.033717 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5f05-f3e0-4c8b-abb9-e95c40921aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:07.033997 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5f05-f3e0-4c8b-abb9-e95c40921aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:07.034319 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb5f05-f3e0-4c8b-abb9-e95c40921aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:07.035034 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb5f05-f3e0-4c8b-abb9-e95c40921aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:07.036704 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb5f05-f3e0-4c8b-abb9-e95c40921aa6 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:07.048072 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1441/2899] 10.222.0.33 () {58 vars in 1121 bytes} [Mon Jun 20 11:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 11:28:07.052297 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435869bb-1bec-41df-aa14-915ab5a3fec4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:07.062913 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435869bb-1bec-41df-aa14-915ab5a3fec4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:07.067812 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19146752-6329-43ae-8779-7c2d0dd9bb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:07.068741 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19146752-6329-43ae-8779-7c2d0dd9bb50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:07.069015 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19146752-6329-43ae-8779-7c2d0dd9bb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:07.069291 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19146752-6329-43ae-8779-7c2d0dd9bb50 admin admin] PATH_INFO: `/v3/projects/44bccb33574a441ca04a3da2d1ed1585/users/6d5b7c9509c8442d829c28380dbcb8a7/roles/3e46d43e30ef494087da2c44f7ca702f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:07.089027 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19146752-6329-43ae-8779-7c2d0dd9bb50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44bccb33574a441ca04a3da2d1ed1585, user_id=6d5b7c9509c8442d829c28380dbcb8a7, role_id=3e46d43e30ef494087da2c44f7ca702f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:07.092972 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19146752-6329-43ae-8779-7c2d0dd9bb50 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:07.111914 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1459/2900] 10.222.0.33 () {60 vars in 1482 bytes} [Mon Jun 20 11:28:07 2022] PUT /identity/v3/projects/44bccb33574a441ca04a3da2d1ed1585/users/6d5b7c9509c8442d829c28380dbcb8a7/roles/3e46d43e30ef494087da2c44f7ca702f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:28:07.123701 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b2aea-a42d-472b-940f-5417db534d93 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:07.123981 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b2aea-a42d-472b-940f-5417db534d93 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:07.124250 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b2aea-a42d-472b-940f-5417db534d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:07.155775 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-be8b2aea-a42d-472b-940f-5417db534d93 None None] MFA Rules not processed for user `6d5b7c9509c8442d829c28380dbcb8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:28:07.156435 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-be8b2aea-a42d-472b-940f-5417db534d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:07.231326 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1442/2901] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:28:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:28:07.242451 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19146752-6329-43ae-8779-7c2d0dd9bb50 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:07.244597 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-19146752-6329-43ae-8779-7c2d0dd9bb50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:07.263529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:28:07.316052 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19146752-6329-43ae-8779-7c2d0dd9bb50 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:07.320009 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92008b6-5ffa-4870-86ce-914bbba244de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '841e89a6caef4a089edc0f3b1df2fe9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:07.320662 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92008b6-5ffa-4870-86ce-914bbba244de service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:07.320942 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92008b6-5ffa-4870-86ce-914bbba244de service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:07.321216 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92008b6-5ffa-4870-86ce-914bbba244de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:07.330399 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92008b6-5ffa-4870-86ce-914bbba244de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:07.331855 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92008b6-5ffa-4870-86ce-914bbba244de service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:07.340968 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1460/2902] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:28:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:28:08.736038 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861ca22-1d6d-48e2-99cf-a5afc5957ecc None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:08.736620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861ca22-1d6d-48e2-99cf-a5afc5957ecc None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:08.736620 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861ca22-1d6d-48e2-99cf-a5afc5957ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:08.755862 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-3861ca22-1d6d-48e2-99cf-a5afc5957ecc None None] MFA Rules not processed for user `6d5b7c9509c8442d829c28380dbcb8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:28:08.757079 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3861ca22-1d6d-48e2-99cf-a5afc5957ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:08.765307 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1443/2903] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:28:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 11:28:17.337119 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=20 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:28:18.749960 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:28:24.214744 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee417ca-31c4-45ff-9acb-44d36f9a7b01 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:24.215585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee417ca-31c4-45ff-9acb-44d36f9a7b01 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:24.215585 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee417ca-31c4-45ff-9acb-44d36f9a7b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:24.234529 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1ee417ca-31c4-45ff-9acb-44d36f9a7b01 None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:28:24.235879 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1ee417ca-31c4-45ff-9acb-44d36f9a7b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:24.318461 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1461/2904] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:28:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:28:24.331095 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861ca22-1d6d-48e2-99cf-a5afc5957ecc None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:24.332725 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-3861ca22-1d6d-48e2-99cf-a5afc5957ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:24.408839 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861ca22-1d6d-48e2-99cf-a5afc5957ecc None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:24.414458 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d603473-5474-454f-9559-47442cec5940 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:24.415124 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603473-5474-454f-9559-47442cec5940 service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:24.415403 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603473-5474-454f-9559-47442cec5940 service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:24.415678 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603473-5474-454f-9559-47442cec5940 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:24.425543 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d603473-5474-454f-9559-47442cec5940 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:24.428207 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d603473-5474-454f-9559-47442cec5940 service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:24.440365 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1444/2905] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:28:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:28:25.021698 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19904bc-0ae5-4593-86da-63e854cd457c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:25.021991 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19904bc-0ae5-4593-86da-63e854cd457c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:25.022311 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19904bc-0ae5-4593-86da-63e854cd457c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:25.044765 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f19904bc-0ae5-4593-86da-63e854cd457c None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:28:25.046004 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f19904bc-0ae5-4593-86da-63e854cd457c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:25.055171 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1462/2906] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jun 20 11:28:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:28:25.069692 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d603473-5474-454f-9559-47442cec5940 service swift] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:25.086734 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d603473-5474-454f-9559-47442cec5940 service swift] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:25.092783 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163c51-5e38-4e27-8163-eb953ef5acfb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:25.093695 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163c51-5e38-4e27-8163-eb953ef5acfb service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:25.094122 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163c51-5e38-4e27-8163-eb953ef5acfb service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:25.094390 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163c51-5e38-4e27-8163-eb953ef5acfb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:25.103450 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2163c51-5e38-4e27-8163-eb953ef5acfb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:25.105052 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2163c51-5e38-4e27-8163-eb953ef5acfb service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:25.115777 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1445/2907] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:28:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:28:25.328098 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b832ab9-8579-4e27-b964-d634e75283be None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:25.328648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b832ab9-8579-4e27-b964-d634e75283be None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:25.328648 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b832ab9-8579-4e27-b964-d634e75283be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:25.352254 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2b832ab9-8579-4e27-b964-d634e75283be None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:28:25.354116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2b832ab9-8579-4e27-b964-d634e75283be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:25.370988 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1463/2908] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Jun 20 11:28:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:28:25.388832 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163c51-5e38-4e27-8163-eb953ef5acfb service swift] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:25.402171 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163c51-5e38-4e27-8163-eb953ef5acfb service swift] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:25.409036 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6fb02-c7d8-477a-8050-92969305aee5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:25.409718 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6fb02-c7d8-477a-8050-92969305aee5 service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:25.410004 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6fb02-c7d8-477a-8050-92969305aee5 service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:25.410313 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6fb02-c7d8-477a-8050-92969305aee5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:25.420080 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6fb02-c7d8-477a-8050-92969305aee5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:25.421733 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6fb02-c7d8-477a-8050-92969305aee5 service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:25.431279 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1446/2909] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:28:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:28:25.784969 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b03fd5-2f0e-4af6-8e0b-268dde02291c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:25.785252 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b03fd5-2f0e-4af6-8e0b-268dde02291c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:25.785530 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b03fd5-2f0e-4af6-8e0b-268dde02291c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:25.805012 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-26b03fd5-2f0e-4af6-8e0b-268dde02291c None None] MFA Rules not processed for user `aca3b45e80764c3b8254e24d09ff0d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:28:25.806282 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-26b03fd5-2f0e-4af6-8e0b-268dde02291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:25.814730 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1464/2910] 10.222.0.33 () {58 vars in 967 bytes} [Mon Jun 20 11:28:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:28:25.828185 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6fb02-c7d8-477a-8050-92969305aee5 service swift] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:25.840062 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6fb02-c7d8-477a-8050-92969305aee5 service swift] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:25.849673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd72326-1f6f-4eb5-92e3-eac52f95b16e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c133005ca8d4478be8ff4acad5e88cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:25.849673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd72326-1f6f-4eb5-92e3-eac52f95b16e service swift] REQUEST_METHOD: `GET` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:25.849673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd72326-1f6f-4eb5-92e3-eac52f95b16e service swift] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:25.849673 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd72326-1f6f-4eb5-92e3-eac52f95b16e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:25.863890 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd72326-1f6f-4eb5-92e3-eac52f95b16e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:25.865867 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd72326-1f6f-4eb5-92e3-eac52f95b16e service swift] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:25.876509 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1447/2911] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Jun 20 11:28:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 11:28:28.262194 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b03fd5-2f0e-4af6-8e0b-268dde02291c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:28.337986 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b03fd5-2f0e-4af6-8e0b-268dde02291c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:28.342946 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:28.343837 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:28.344118 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:28.344388 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] PATH_INFO: `/v3/users/6d5b7c9509c8442d829c28380dbcb8a7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:28.345828 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d5b7c9509c8442d829c28380dbcb8a7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:28.347836 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:28.406735 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d5b7c9509c8442d829c28380dbcb8a7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:28.414999 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d5b7c9509c8442d829c28380dbcb8a7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:28.415620 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d5b7c9509c8442d829c28380dbcb8a7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:28.416224 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d5b7c9509c8442d829c28380dbcb8a7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:28.429692 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d5b7c9509c8442d829c28380dbcb8a7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:28.437870 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:28:28.443936 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1465/2912] 10.222.0.33 () {58 vars in 1223 bytes} [Mon Jun 20 11:28:28 2022] DELETE /identity/v3/users/6d5b7c9509c8442d829c28380dbcb8a7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:28:28.449665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3516-3911-4abc-8319-9582f82c7bd6 None None] REQUEST_METHOD: `POST` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:28.450572 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3516-3911-4abc-8319-9582f82c7bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:28.450851 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3516-3911-4abc-8319-9582f82c7bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:28.478179 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.auth.core [None req-a6ff3516-3911-4abc-8319-9582f82c7bd6 None None] MFA Rules not processed for user `52894fe1eaba4d88aa400c34b4d5791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80132) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 11:28:28.480284 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a6ff3516-3911-4abc-8319-9582f82c7bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:28.567752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1448/2913] 10.222.0.33 () {58 vars in 957 bytes} [Mon Jun 20 11:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 11:28:28.580619 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:28.582116 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:28.642814 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4043a-3b30-4c74-8758-e43f7b1f4382 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:28.646534 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c68b24-5fd4-48dc-bc6d-f2fc838fd053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd73fe529c7d342f895aabcb9a39aba78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:28.647202 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c68b24-5fd4-48dc-bc6d-f2fc838fd053 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:28.647488 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c68b24-5fd4-48dc-bc6d-f2fc838fd053 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:28.647767 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c68b24-5fd4-48dc-bc6d-f2fc838fd053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:28.657853 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c68b24-5fd4-48dc-bc6d-f2fc838fd053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:28.659346 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c68b24-5fd4-48dc-bc6d-f2fc838fd053 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:28.669944 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1466/2914] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:28:28.859479 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3516-3911-4abc-8319-9582f82c7bd6 None None] Authenticating user token {{(pid=80132) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:28:28.861473 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.fernet_utils [None req-a6ff3516-3911-4abc-8319-9582f82c7bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80132) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:28:28.874388 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3516-3911-4abc-8319-9582f82c7bd6 None None] Validating token access rules against request {{(pid=80132) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:28:28.879587 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52894fe1eaba4d88aa400c34b4d5791b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f76a7fac220343628fdd3080ad89317e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80132) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:28:28.880469 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:28:28.880752 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:28:28.886320 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] PATH_INFO: `/v3/projects/44bccb33574a441ca04a3da2d1ed1585` {{(pid=80132) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:28:28.887665 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44bccb33574a441ca04a3da2d1ed1585)` {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:28:28.889654 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] RBAC: Authorization granted {{(pid=80132) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:28:28.942009 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:28:28.945990 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44bccb33574a441ca04a3da2d1ed1585', 'request_id': 'req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:28.954219 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44bccb33574a441ca04a3da2d1ed1585', 'request_id': 'req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:28.960136 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44bccb33574a441ca04a3da2d1ed1585', 'request_id': 'req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675'} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:28.971703 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] The token cache is being invalidate because project 44bccb33574a441ca04a3da2d1ed1585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80132) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 11:28:28.971992 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:28.972651 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG keystone.notifications [None req-d59ce12c-b3be-4a92-9ce9-b0b91cdfc675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80132) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 11:28:28.974287 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: [pid: 80132|app: 0|req: 1449/2915] 10.222.0.33 () {58 vars in 1232 bytes} [Mon Jun 20 11:28:28 2022] DELETE /identity/v3/projects/44bccb33574a441ca04a3da2d1ed1585 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 11:28:38.664885 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=27 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:28:38.968381 dv-cinder-846146-3-iscsi devstack@keystone.service[80132]: DEBUG dbcounter [-] [80132] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80132) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 11:32:52.340854 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c68b24-5fd4-48dc-bc6d-f2fc838fd053 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 11:32:52.342507 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a6c68b24-5fd4-48dc-bc6d-f2fc838fd053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:32:52.419420 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c68b24-5fd4-48dc-bc6d-f2fc838fd053 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 11:32:52.487948 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ca845-645d-4891-9618-9bb3b2e708f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bff19e37fa2243fda914f7f4e662c8f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c904620b9191403ca36d0aca020db817', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 11:32:52.488499 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ca845-645d-4891-9618-9bb3b2e708f4 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 11:32:52.488785 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ca845-645d-4891-9618-9bb3b2e708f4 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 11:32:52.489062 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ca845-645d-4891-9618-9bb3b2e708f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 11:32:52.490974 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b60ca845-645d-4891-9618-9bb3b2e708f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 11:32:52.503447 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ca845-645d-4891-9618-9bb3b2e708f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 11:32:52.505081 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ca845-645d-4891-9618-9bb3b2e708f4 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 11:32:52.516294 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1467/2916] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Jun 20 11:32:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 11:33:02.511350 dv-cinder-846146-3-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=58 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}